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I. INTRODUCTION 


The Fleet Management System-Real Time (FMS-RT), a relatively 
new computer system, was introduced to the fleet on 1 January 1988. 
The FMS-RT system is a portable stand-alone system that is delivered 
to fleet units going through overhauls or major industrial availabilities 
of greater than four months in duration. The overhauls and availabili- 
ties that utilize FMS-RT are: Complex Overhauls (COHs), Regular Over- 
hauls (ROHs), Drydock or Extended Docked Scheduled Restricted 
Availabilities (DSRAs and EDSRAs), Drydock or Extended Docked 
Phased Maintenance Availabilities (DPMAs and EDPMAs), and any 
other availability requested by the type commander (TYCOM). 

The FMS-RT system replaced the fleet Ship's Force Overhaul 
Management System (SFOMS). The system was required due to the 
Navy's decision to eliminate the use of the INFONET system, the 
backbone telecommunications network for SFOMS. Naval Sea Systems 
Command Automated Data Systems Activity (SEAADSA) was given two 
years to design and install FMS-RT hardware and software, while Naval 
sea Systems Command Detachment Planning Engineering Repairs and 
Alterations (PERA CRUDES) prepared the training program. 

The objective of the FMS-RT system is to optimize the use of the 
ship's force personnel and maintenance resources during overhauls. 
The goal of the system is to increase performance so that overhauls 


will be shorter in duration and at a lesser cost to the government. 


Of the many questions that could be addressed surrounding the 
FMS-RT system, two questions drive this research: (1) To what extent 
does the curriculum teach people to efficiently and effectively utilize 
the FMS-RT system in the management of resources in an overhaul 
environment? and (2) To what extent is the FMS-RT system usable by 
the end user? “End user” in this case means the person who directly 
interfaces with the computer and the people who utilize the input and 
output reports. 

The premise of this thesis is that, with a quality software package 
and good training, the FMS-RT system would increase ship's force 
work performance. We are not testing this relationship; we are merely 
making the assumption that with good quality software and training, 
one can better utilize the FMS-RT system and thus increase 
performance. 

It is therefore imperative that we define exactly what we are 
talking about in terms of training and software quality. We will begin 
with the issue of training. 

What is training? What is meant when we say that we train people 
in the use of computer systems? Webster’s Ninth New Collegiate Dic- 
tionary indicates that to train means "to teach so as to make fit, quali- 
fied, or proficient." [Ref. l:p. 1251] Training means “the teaching, 
drill, or discipline by which powers of mind or body are developed: 
education.” [Ref. 2:p. 2424] 

Training can be assessed in many different ways. For example, we 


could investigate who conducts the training and how well they 


conduct it, what methods are used and how appropriate and successful 
they are, and what the content of the training curriculum is and 
whether it contains all the necessary and sufficient information the 
learner needs to know and master. The focus of this thesis is on the 
contents of the curriculum, or what we refer to as the training cur- 
riculum- the lecture topics and the supplemental information con- 
tained in hand-outs, diagrams, and other documents. 

We chose the content of the training curriculum because it is 
established and we have experts in shipboard overhauls who can 
review the materials to ascertain if what is being covered is what 
should be covered in the limited training time available. Training con- 
tent is an issue because the number of training phases have decreased 
with the installation of the FMS-RT system, and because overhauls are 
of such a relatively short duration (four months to a year) that we can- 
not afford a large learning curve on a system if we are to get increased 
performance from that system. 

Our second issue is that of software. What is the quality of the 
FMS-RT software? Numerous criteria exist to judge the quality of soft- 
ware. For example, we see in Pressman's book on Software Engineer- 
ing [Ref. 3:pp. 452-463] that there are various factors that can be used 
to evaluate the success of any software. The McCall, et al., model [Ref. 
3:p. 454] illustrates the factors, their use, and some of the metrics 


used for their measurement (see Figure 1). 


Criteria Factor Use 


Operability 
AS Usobility 
Input/output volume 
Integrity 
Input/output rate 
Efficiency 
Storage efficiency 
Execution efficiency 
Reliability 
Maintainability 
Testability 
Flexibility 


Traceability 
Reusability 


Completeness 
— 
Portability 
Interoperability 


S 
Figure 1. McCall, et al., Model 
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McCall's model has three uses: product operation, product revi- 
sion, and product transition. These uses translate to factors which 
have measurable criteria or metrics. Since FMS-RT is a new software 
product, revision and transition issues are of lesser importance than 
product operation. The five factors of product operation are usability, 
integrity, efficiency, correctness, and reliability. Efficiency and 
integrity are controlled by the FMS-RT user through restricted access 
to the system and there is little need to optimize the computer's effi- 
ciency. The factors most important to the researcher are usability, 
reliability, and correctness. Usability is initially important because if 
we cannot initially use a system we do not need to worry about issues 
like reliability or correctness. 

This study evaluates the usability factor. Can the user easily oper- 
ate the FMS-RT system? Usability is the quality or state of being use- 
able. Useable means capable of being used; convenient and practicable 
for use [Ref. 1:p. 1299]. We investigate usability because at present the 
Navy has very little actual user involvement in system development. 
This lack of user involvement often leads to systems that work but 
which, due to their complexity, cannot be or are not used proficiently 
(he Teer. 

To summarize our research, this paper investigates two overlap- 
ping issues— training curriculum and usability. We assume that, given 
an appropriate training curriculum and a usable piece of software, we 
should get increased performance from our ship's work force during 


the overhaul. Ultimately, recommendations are made to improve the 


FMS-RT system based on what the actual users have to say about the 
usability of the system and on what experts in shipboard overhauls 


have to say about the training curriculum. 


A. THESIS OVERVIEW 
1l. Chapter II: Background 


a. Background of Overhaul Process and Ship's Force 
Role 


This section provides a brief description of the funding 
for an industrial availability, the process of preparing a ship's alter- 
ation and repair package, and the process of distributing work 
between a contractor and the ship's force personnel. A general time 
line of the overhaul planning process is provided. 


b. Background of Old Overhaul Management Systems 
and Problems 


This section is divided into three parts. The first part 
describes the old manual tracking system and its problems. The sec- 
ond part describes the Ship's Force Overhaul Management System 
(SFOMS) and the problems with the system. The last part of this 
chapter illustrates two problems with developing any computer-based 
ship's force tracking system— infrequent use of the system and com- 
plexity of the system to the user. 

2. Chapter III: Description of FMS-RT 
This section gives a general description of the FMS-RT hard- 
ware and software as well as a description of FMS-RT as a management 


information system (MIS). 


3. Chapter IV: Methodology 
This section describes what data was collected, from the data 
was collected, and how it was collected. 
4. Chapter V: Results 
This section contains the data. The collected data is split 
between the training curriculum— current and optimal— and the usabil- 
ity survey results. 
5. Chapter VI: Analysis 
This section is an analysis of the results. We compare curric- 
ulum contents and identify usability problems. Interpretation of the 
data is presented to support these assessments. 
6. Chapter VII: Recommendations and Conclusion 
In addition to summarizing proposed enhancements to the 
FMS-RT system, recommendations are made for additional research 
into areas of study related to this thesis. 
7. Appendices 
Appendices are provided to assist the reader and to support 


textual material. 


II. BACKGROUND 


A. BACKGROUND OF OVERHAUL PROCESS AND SHIP'S FORCE 
ROLE 


1. Overhaul Funding 

The funds required to overhaul a ship run well into the mil- 
lions of dollars. The current cost to overhaul one Spruance Class 
Destroyer is about $25 million. It is about $18 million to overhaul one 
frigate. The Navy overhauls ships based upon their class and that 
class’s maintenance schedule. Therefore, on an annual basis we must 
set aside an enormous sum of money just for the maintenance and 
upkeep of our fleet. 

Two groups conduct annual budgeting for overhauls. First, the 
type commander [TYCOMs are Commander Naval Surface Force Pacific 
(COMNAVSURFPAC) and Commander Naval Surface Force Atlantic 
(COMNAVSURFLANT)] budgets funds for the ships under his control. 
He funds general upkeep, maintenance, and repairs. Second, the Naval 
Sea Systems Command (NAVSEA) budgets funds for technical research 
and alterations designed to improve the capabilities of our ships. 
These two major claimants control the funds that are required to 
overhaul our ships. The Chief of Naval Operations must approve all 
alterations made on each ship, but the alterations are financed by 
NAVSEA. 

The amount spent per ship and the number of ships over- 


hauled are determined by the amount of money the Secretary of 


Defense appropriates to the Chief of Naval Operations for each of these 
major claimants after the annual budget has been passed. A long-range 
ships schedule, which includes major industrial availabilities, aids 
these two commands and schedules ships for maintenance. An over- 
haul requires substantial financial resources from both the TYCOM and 
NAVSEA. The following section describes from where the financial 
estimates for the total cost of an overhaul come. 
2. Overhaul Planning Process 

Planning for an industrial availability requires the interaction 
of many programs, systems, and activities. The objective of this plan- 
ning process is to define a work package, allocate available resources, 
obtain required materials, and schedule work. Because of the numer- 
ous variables involved in planning, this process is inherently difficult 
to manage. 

A ship’s overhaul package consists of two major types of 
work- alterations and repairs. Development of the two work packages 
proceeds along different paths through different chains of command, 
but both must be merged before the overhaul for review and initial 
planning. 

The repair work package is generally developed and work 
accomplished by the following process: 

a The TYCOM tasks the appropriate Planning and Engineering for 
Repair and Alterations Command (PERA) to prepare the Ship's 
Alteration and Repair Package (SARP). 


b. PERA schedules the ship to conduct a Pre-Overhaul Test and 
Inspection (POT&I). Generally, POT&I is scheduled about nine 
months prior to overhaul commencement. 


PERA conducts a review of the Current Ship's Maintenance 
Project (CSMP), along with the results of the most recent annual 
INSURV inspection and POT&I to ascertain all the discrepancies 
with the ship. 


. PERA prepares the SARP, estimates the cost of overhauling the 


ship, and submits the package to the TYCOM. 

TYCOM screens the SARP and approves the work package. 
Contracts are awarded, material is ordered and staged for the 
overhaul, jobs are divided between shipyard or contractor and 
either ship's force or an intermediate maintenance activity 
(IMA), e.g., a SIMA, MOTU, or a tender. 


Overhaul begins. 


h. TYCOM approves new work as appropriate throughout the 


overhaul. 
Overhaul ends and ship departs. 


Development of the alteration package consists of the follow- 


ing tasks: 


a 


Planned alterations are listed in the Fleet Modernization Program 
(FMP). Alterations applicable to the ship are reviewed. 


Approximately one year prior to the overhaul, PERA is tasked 
with advance planning for specific alterations. 


. The alterations are incorporated into the SARP and cost esti- 


mates are forwarded to the appropriate command after the alter- 
ations are ship-checked. 


. The CNO-funded alteration package is provided to TYCOM about 


six months prior to the overhaul. 


. The CNO alteration package is modified at the pre-arrival confer- 


ence and the TYCOM-funded alterations are firmed. 
Overhaul begins. 


Overhaul ends and ship departs. 
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Developing the overhaul package by these processes results in 
certain problems. Typically, the SARP is prepared late and is of ques- 
tionable quality because of late or incomplete information, access to 
the ship is restricted due to conllict with operational schedules, and 
each overhaul is treated virtually as if a ship had never been through an 
overhaul before. 

These problems are addressed by the pre-arrival conference, 
where members of the ship, TYCOM, PERA, Supervisor of Shipbuild- 
ing, Conversion, and Repair (SUPSHIPS is the group that does con- 
tract administration and accepts work for the government), and the 
IMAs bring out problems within the SARP. Conflicts over who should 
do certain work because of either the scope of the work or expertise 
required are resolved. Problems with omission of jobs, as well as 
problems with improper job descriptions or misunderstood job 
descriptions, are corrected. 

The output of this process is a reliable SARP that can be used 
to solicit bids from contractors and to begin the initial planning. From 
this effort, the ship has a baseline from which to proceed into the 
detailed planning of the overhaul, including the baseline for the plan- 
ning and implementation of the FMS-RT system. It is critical to realize 
that the FMS-RT system only tracks ship's force work and work 
screened to IMAs, that is, work to be done by the Navy, not contractor 


work. 
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Figure 2 is a summary time line of the overhaul process. This 
time line varies slightly depending on the type of overhaul or availabil- 


ity and the ship type. 


B. BACKGROUND OF OLD MANAGEMENT SYSTEMS AND 
PROBLEMS 


1. Manual Ship's Force Tracking System 

This system refers to the non-automated tracking system 
used by ships prior to the Ship's Force Overhaul Management System 
(SFOMS) that will be discussed later in this chapter. This manual sys- 
tem is hard to describe because it varied from ship to ship and the 
overhaul process was different than it is today. The ships' personnel 
worked in conjunction with the shipyard. They were tasked with jobs 
by the contract administrator, SUPSHIPS, and most overhauls were 
conducted at Navy shipyards. 

Today, more work is contracted out to private contractors. 
The Office of Management and Budgeting Circular Number A-76 
[Ref. 4] established the policy of not having the government compete 
over contracts that can be accomplished by commercial sources. This 
policy emphasizes having commercial activities do as much as possible 
so the size of our military force can be controlled and so we can train 
our military to meet their primary mission. Contracts are also given to 
create and maintain a strong base of shipbuilding and ship repair 
facilities in case of mobilization. We hold the commanding officer of 


the ship responsible for the overhaul and the employment of 


J. 
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Figure 2. General Overhaul Time Line 
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his resources, both financial and personnel. Prior to 1975, when this 
system was put in place, overhauls were done by naval shipyards with 
the crew and the shipyard personnel working together. The 
commanding officer of the ship and the project manager of the 
overhaul worked closely to coordinate the work to be done in the 
overhaul. 

The manual system was either set up by the ship or, in some 
cases, the shipyard would provide a work plan to the ship in the form 
of a PERT or CPM-tvpe chart. Turban and Meredith [Ref. 5:pp. 319- 
377] explain the general concepts of program management. PERT is 
the Program Evaluation Review Technique and CPM is the Critical 
Path Management Technique. The difference between these two pro- 
gram management techniques is the way in which activity durations 
are estimated. PERT utilizes a form of weighted average or a proba- 
bilistic approach and CPM uses only one estimate for duration of an 
activity. In addition, CPM allows an explicit cost estimate in addition 
to time. PERT is basically a tool for planning and controlling time. 

As Turban and Meredith point out, PERT and CPM force 
management to plan in detail and to define what must be done to 
insure timely completion of the project. These techniques are easily 
understood because they provide a method of visualizing the entire 
project. The Navy requires submission of either a PERT or CPM plan 
with bids from contractors on ship overhauls (see reference for fur- 


ther details). 
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A problem with the manual system is that the unit going 
through overhaul has little control over the format of information. 
Each contractor or shipyard had its own system for job tracking. The 
ships had different tracking methods internally. The internal systems 
sometimes changed from department to department or from division 
officer to division officer. Some ships had them on pocket notebooks, 
others had standard forms made up for tracking purposes. In short, 
there was no standard established in the Navy— internal reports could 
be standardized if the command emphasized standardization. 
However, internal and external reports were not standard, making 
decision making and problem identification very difficult. These infor- 
mation-handling differences often caused duplication of effort, which 
drove up the cost of overhauls and increased their duration. 

We still see the use of manual tracking today as a back-up to 
the current computer-based systems used in the Navy. While aboard 
USS Hewitt, the engineer tracked key jobs for his pocket notebook 
(wheelbook) and had standard forms for each ship's force job in his 
department; these were updated twice a week. He had this manual 
tracking system because his forms had a remarks column with more 
detailed information than the reports generated by the computer- 
based system. 

2. The Ship's Force Overhaul Management System 

In 1975, Pearl Harbor Naval Shipyard initiated an Overhaul 

Improvement Program. The goal of the program was to make more 


efficient use of resources (personnel and money). After a review of the 
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maintenance data available at the time and 12 alternatives for a man- 
agement information system (MIS) that could track ship's force 
work,the SFOMS tracking system was adopted. The purpose of this 
system was to eliminate the duplication of effort and to make more 
efficient use of ship’s force personnel. The end result, one hoped, 
would be an overhaul of shorter duration and at a cheaper cost to the 
government. 

The system was a batch data-processing system. The database 
was updated weekly using a transaction file. The system was designed 
using a central host computer (mainframe computer), feeding changes 
from a microcomputer through the use of the Computer Systems Cor- 
porations Information Network (INFONET), the backbone telecom- 
munications network. 

The problem with the SFOMS system was that, as of 1 Jan- 
uary 1988, the Navy stopped utilizing the INFONET. Additionally, 
reports were frequently late because of the usual process of submitting 
a single request for an update with reports. Up-to-date reports and an 
updated database could only be obtained by request. The process con- 
sisted typically of submitting the updates on Friday. The database was 
first updated and then the reports were produced and delivered to the 
ship. This report turnaround process meant that changes needed to be 
in by Thursday, and reports would be back to middle management by 
Tuesday, thus allowing just three days for problem identification, 


scheduling corrections, and other decision-making processes. 
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Problems with data correctness also existed because only 
weekly updates were available. In addition, the report formats did not 
answer all the questions that the commanding officer had and there 
was no “what if” query capability. The inputs were substantially driven 
by paperwork. The input forms were filled out and submitted for 
review up the chain of command, with only one or two people 
inputting changes to the database. This allowed forms to be lost or 
data to be entered incorrectly. The system was complex, functional, 
and cumbersome. However, the major reason for its downfall was the 
loss of the INFONET and the emergence of powerful microcomputer- 
based systems that could do the same job in a stand-alone capacity. 

3. Problems Computer-Based Systems Must Overcome 

In the area of computer-based tracking systems for ship's 
force work, there are two problems- infrequent use and system com- 
plexity. Ships go through overhauls and other industrial availabilities 
based on a maintenance cycle for the particular class of ship. These 
major upkeep periods are every three to five years. Therefore, we use 
this computer tracking system very infrequently. Because we do not 
utilize the system regularly, we must be trained on the system before 
we can be expected to utilize it efficiently and effectively. 

Additionally, the person who interfaces directly with the 
computer is not trained on computers and has other responsibilities 
aboard ship. For the most part, Navy ships do not have people trained 
int eh use of microcomputers. Were we to stereotype the user, we 


would say he was a responsible Second Class Petty Officer up to a Chief 
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Petty Officer; he is not in charge ofla workecentermhe hasta 
background in electronics, and he has only an “average American 
knowledge” of using a computer. He did not get formal training on 
computers prior to using this system, and he does not own his own 
personal computer. He has no programming skills. In short, he has a 
limited knowledge of computers. 

Therefore, we cannot have complex systems that are not easy 
to use. Our system needs to be extremely forgiving and very “user- 


friendly.” 
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III. DESCRIPTION OF FMS-RT 


A. HARDWARE 
The FMS-RT system was designed to operate on an AT-compatible 
microcomputer that consists of the following: 
e an AT keyboard layout 


° a minimum of 20 megabytes of internal fixed disk (access time of 
40 MS or less) 


a 360-kilobyte diskette drive 

e a high-resolution composite graphics monitor 

e a Hercules-compatible composite color video card 
e a graphics-capable printer (minimum 200 CPS) 


e a minimum of one megabyte of memory (expandable to 2.6 
megabytes) 


e atape streamer back-up 

e an internal date and time clock (battery powered/back-up) 
e MS-DOS version 2.1 or greater 

° an asynchronous communications interface 

e an 80287 math co-processor 

e a serial port (one) 

e a parallel port (one) 

e two 8/16 bit expansion slots (unused) 

e an IBM-compatible graphics card 


e an electrical surge supressor 


E9 


Generally speaking, FMS-RT uses the Zenith-248 computer with 
Zenith monitors and keyboards and the Alps-P2000G or P2100G 


printer. 


B. SOFTWARE 

The program life-cycle manager for the software is Naval Sea Sys- 
tems Command Automated Data Systems Activity (SEAADSA). The 
software was programmed in the high-level computer language COBOL 
(Common Business Oriented Language). COBOL's strength is in busi- 
ness applications and it interfaces well with other languages. Since 
FMS-RT is a project-management tool, COBOL is an understandable 
programming language selection. 

The software consists of eight standard five and one-quarter inch 
(5.25) floppy disks (double sided, double density). The disks contain 
156 fils. The files are functional processes, which indicates that the 
software used a modular design concept. Also, the file sizes and order 
indicate that during design, functional decomposition was utilized to 
break processes down into programmable modules. 

The use of 156 files will make debugging, troubleshooting, and 
enhancements very difficult due to the large number of module inter- 
faces. A change to one module will require investigation into the 
impact of that change on all the modules with which the module to be 
changed interacts. This means a great deal of testing will be done to 


verify the impact of software changes. 
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The software has limited sort features. The user can only get 
sorted information on selected fields, not on any field of his choice. 
This prohibits user query capability and "what if"-type questions. 

For further descriptive information on the FMS-RT system, read- 


ers should consult the FMS-RT Operator's Guide [Ref. 6]. 


C. THE SYSTEM 

FMS-RT was introduced to the fleet on 1 January 1988. It was 
designed as a portable stand-alone management information system 
(MIS). The purpose of the system was to track ship's force work dur- 
ing overhauls and availabilities. 

A portable stand-alone system means that the computer system is 
not linked to other systems via a network and the computer system is 
controlled by the user at the installation site. The real-time part of 
FMS is that updates and report generation can be done anytime the 
unit chooses. It does not use the typical computer terminology defini- 
tion of real time, which is that the output of the computer is severely 
time constrained, so the computer processing time must be extremely 
efficient to meet the time constraint of obtaining output. An example 
of this is a fire control solution for a weapons system, which must be 
real-time processing to be of any value. 

The basic process in computer operations in input-process-out- 
put. This means that when a computer is operating and given an input, 
a process is triggered which has the computer manipulate data and it 


produces an output. The top of Figure 3 illustrates this principle. 
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Figure 3. FMS-RT Described by Head’s MIS Triangle 
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Why is FMS-RT an MIS? What is an MIS? A triangle was used by 
Robert Head in the late 1960s as a visual model to characterize MIS in 
a broad, comprehensive sense [Ref. 7]. MIS characteristics include an 
information focus aimed at the middle-level managers, structured 
information flows, an integration of low-level electronic data-process- 
ing functions, and inquiry and report generation, usually with a 
database. These are the same characteristics found in the FMS-RT 
system. Figure 3 uses Head's triangle. 

The vertical axis indicates the level of management (in our figure, 
the chain of command). The horizontal axis indicates functional 
departments. The figure illustrates that there is a structured flow of 
information. The flow proceeds up the chain of command to the 
department-head level before being entered into the FMS-RT systems. 
The systems process the input data and the figure illustrates that most 
output repots are aimed at the middle-level manager. Structured flow 
of information aimed at middle-level managers in report form is char- 
acteristic of MISs. Many low-level data-processing functions are 
invoked to manipulate the data. For these reasons, we see the FMS-RT 
system as an MIS. 

The input and output forms referred to in Figure 3 are found in 
the FMS-RT Administrative Manual [Ref. 8] and copies are provided for 
the reader in Appendix B. 

In summary, FMS-RT can be described as a project management 
tool. A project management tool tracks many independent job activi- 


ties by sequencing those that require sequencing and tracking them 
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In summary, FMS-RT can be described as a project management 
tool. A project management tool tracks many independent job activi- 
ties by sequencing those that require sequencing and tracking them 
all from start to finish. FMS-RT tracks ship's force jobs from the start 
of the overhaul to completion. It links jobs to key events in the over- 
haul process. For example, if job A needs to be done while the ship is 
in drydock, and docking and undocking the ship are key events, then 
job A will be linked to the key event of undocking the ship so that any 
attempt to schedule job A after undocking receives a flag to indicate 


that there is a problem with the proposed change. 
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IV. METHODOLOGY 


A. DESIGN 

The two research questions are “To what extent does the cur- 
riculum teach people to efficiently and effectively utilize the FMS-RT 
system? and “To what extent is the FMS-RT system usable by the end 
user? In this section, we will lay out what data were collected to 
answer these questions. We will begin with the training curriculum 
COmMent. 

A way to measure training content is to do a comparative analysis 
of the current curriculum and an ideal curriculum. There were four 
steps in this type of research design. First, the current curriculum 
content was ascertained by interviewing the trainer and by having the 
researcher participate in the training program. A comparison of the 
two reports— one from the trainer and one from the researcher— 
revealed no differences in the content of the training program. Sec- 
ond, three experts were identified: an expert from within the Navy, an 
expert from outside the Navy, and an expert from SUPSHIPS who is 
the Navy-to-contractor interface. Third, through a structured inter- 
view process, the experts described what they consider the ideal cur- 
riculum content to be. The last step was a comparison of what the 
experts identified as the ideal curriculum and what the researcher and 


trainer identified as the current curriculum. 
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There are three steps in the research design to evaluate the soft- 
ware usability. First, since the users were geographically dispersed, 
the researcher decided to survey major users. Second, the survey was 
constructed. The survey was modeled after Baroudi and Orlikowski's 
short-form measure of user information satisfaction [Ref. 9]. This sur- 
vey was used because Baroudi and Orlikowski demonstrated that the 
survey form used valid measures (the 13 factors had Cronbach's alpha 
at .84 and above). 

In our survey, the first two questions establish the level in the 
chain of command of the user and his experience in the overhaul pro- 
cess. Questions 3 through 10 are the seven factors from Baroudi and 
Orlikowski's survey that applied to the FMS-RT system. We chose only 
the seven that applied to the use of the system and rejected the six 
factors relating to the system design because our users were not 
involved in design. Questions 11 through 17 are specific to FMS-RT: 
Question 11 measures the average number of changes in data per 
week, Questions 12 through 16 are specific “yes or no” type questions 
regarding already-planned enhancements and the usefulness of the 
FMS-RT system, and Question 17 is an overall evaluation of the 
FMS-RT system. Question 18 is used to determine on which coast the 
training was conducted. [PERA (CRUDES) is responsible for training, 
but currently contracts it out to Envision Corporation in Chula Vista, 
California, for all west-coast (COMNAVSURFPAC) ships and to Stellar 


Corporation in Norfolk, Virginia, for all east-coast 


PAS 


(COMNAVSURFLANT) ships.] Question 19 is an open-ended question 
regarding enhancements or problems with FMS-RT. 

A copy of the cover letter to the commanding officers of the ships 
to be surveyed, the cover letter to the participants, and the survey are 


contained in Appendix C. 


B. SAMPLE 

Who do we sample to get this data? To evaluate the training cur- 
riculum, we sample the experts. To evaluate the usability, we survey a 
sample of commands that have utilized the FMS-RT system. 

To ascertain the ideal training curriculum, we sought three points 
of view: a view from within the Navy, a view from outside the Navy 
(outside contractor), and a view from the interface between the Navy 
and the outside (SUPSHIPS). In addition, since experience was a fac- 
tor, we defined an expert as a person who had spent a minimum of 
three years in an overhaul management environment. 

There are numerous people who fit these requirements. There- 
fore, since we wished to interview them, we added the constraint that 
the expert had to be on the west coast (because of travel time and 
funding constraints). Three people met these requirements: 


1. CDR Loeffler, Commanding Officer, USS Hewitt (DD-966), 
presently half-way through an overhaul. He has used SFOMS and 
FMS-RT, and has been through ten major industrial availabilities 
or overhauls. 


2. Mike Rose, Project Manager for Continental Maritime of San 
Diego (CMSD). He was an engineering division officer during a 
carrier overhaul, resigned from the Navy, and has handled six or 
seven projects for CMSD. 
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3. Robert Cordell, SUPSHIPS Project Manager presently responsible 
for the overhaul of USS Hewitt. He is a GS-12 who has worked on 
ship overhauls as a member of the Navy and who, after getting 
out, went to work for SUPSHIPS as a surveyor and progressed up 
to project manager. 

Who do we sample in our usability survey? The commands sur- 
veyed came from a list of cruiser-destroyer (CRUDES) type ships pro- 
vided by the sponsor of this research, PERA (CRUDES). The list 
consisted of 26 ships that, as of June 22, 1988, were currently using 
or had recently finished using the FMS-RT system. Ten survey forms 
were sent to each command to get a cross-section of responses from 
their users. Because FMS-RT is relatively new, 26 CRUDES ships are 
all that have utilized the system. Other ship types use FMS-RT, but this 
study was directed at only CRUDES-type ships. 

To avoid the requirements of the Privacy Act and to get the true 
feelings of the users, our survey was sent to the commands. The com- 
mand was tasked with choosing those people who directly interface 
with the computer and those who utilize the input and output reports. 
Since the command knows who uses the system the most, the selec- 
tion of survey participants was left up to the command and names 


were not requested. 


C. DATA COLLECTION 
The collection of data on the training curriculum involved the 
following steps: 
1. The researcher collected a generic training curriculum outline 


from PERA (CRUDES) that they utilized in evaluating contractor 
personnel on performance of the training contract. 
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. After review of the overhaul process and training curriculum out- 
line, the researcher scheduled a visit to USS Jarrett (FFG-33) for 
Phase 1 FMS-RT training. 


. The researcher participated in Phase I FMS-RT training aboard 
USS Jarrett (FFG-33) from June 13, 1988 to June 16, 1988. 
During this time, the researcher interviewed the trainer, Robert 
Jackson of Envisions Corporation, on the content of the training 
curriculum in both Phases I and II [Ref. 10]. The researcher did 
not participate in Phase II FMS-RT training but covered it during 
the interview. 


. The researcher obtained a copy of all training documentation 
provided to the ship from Mr. Jackson. 


. From this documentation, the researcher prepared questions for 
the expert interviews. The questions follow: 


What is your experience in ship overhauls? How many? 


Based on your experience, what topics do you feel need to be 
covered in an ideal training curriculum? Be specific— for 
example, how you scope a job or the cost and time con- 
straints the unit must meet. 


c. Do you have any opinion on current Navy training curricu- 
lums? If so, are you familiar with FMS-RT or SFOMS? If yes, 
can you give the strengths and weaknesses of their training 
curriculums? 


d. How important is project management in a training curricu- 
lum? At what level in the chain of command should it be 
taught, if it is important? 


e. The emphasis of these questions is to create an ideal training 
curriculum for using the Navy's FMS-RT system and help 
ships perform more efficiently an effectively during an over- 
haul environment. What components of a curriculum do you 
feel are essential? 


f. Is there anything else you have experienced that could be 
used to improve FMS-RT? Any training you have seen that was 
better than what the Navy has now? 


. The researcher conducted the expert interviews [Refs. 11-13]. 
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Data collection for the user survey involved the following steps: 

1. The researcher prepared the survey in early June. 

2. A pilot survey was run on ten users aboard USS Hewitt (DD-966) 
on June 17, 1988. The respondents encountered no problems in 
understanding the survey questions. 


3. The researcher received a listing of ships that have used FMS-RT 
from PERA (CRUDES). Of the 26 ships on the list, since the USS 
Hewitt was done as a pilot, 10 surveys were produced for each of 
the remaining 25 ships. 


4. Responses were accepted until August 30, 1988. The total num- 
ber of respondents to the survey was 120. Of these 120, ten were 
from the pilot and 11 out of the remaining 25 commands 
responded with 110 surveys. Of the 14 commands that did not 
return surveys, two commands reported never using FMS-RT. 
Therefore, 260 surveys were prepared (10 for each of 26 com- 
mands). 20 surveys were sent to two commands that never used 
FMS-RT. This yielded 240 active surveys sent; 120 were received 
for analysis. 

The administration procedure for the survey requested command 
participation. Therefore, participation by the command was voluntary. 
The surveys were handled at a command level, meaning they were 
sent to the command, handed out, and returned by the command. 
Names were not asked for on the survey and an explanation of the sur- 
vey's use was provided to help solicit actual user feelings. No pressure 
was placed on the command to participate or to accelerate the data 
collection effort. For example, no follow-up message was sent, no 
direction from higher levels in the chain of command was sent, and no 
due date was imposed. We suspect that these were the reasons the 
response rate was only 50 percent (120 responses out of 240 active 


surveys). 
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V. RESULTS 


A. CURRENT CURRICULUM 

The FMS-RT training content combines general concepts in proj- 
ect management with specific training in preparing information to be 
input into the FMS-RT system. Phase I FMS-RT training is five days in 
duration and scheduled about six months prior to the commencement 
of the overhaul. The training is conducted at the ship. The topics cov- 
ered in Phase I follow: 


1. Each work center is provided a copy of the FMS-RT Work Center 
Supervisor User's Booklet [Ref. 14]. 


2. Industrial work (man-hours) is defined as man-hours relating 
directly to overhaul work. Non-industrial work is time spent not 
on overhaul work, like paperwork, watch standing, leave, schools, 
and other non-overhaul-related items. In the content of this 
topic, historic data from past ships shows the relationship 
between industrial man-hours versus non-industrial man-hours. 
For example, a work center usually has a 40 percent/60 percent 
split. This indicates that in an eight-hour work day, 40 percent of 
a worker's time is spent on industrial work and 60 percent is 
spent on non-industrial work. 


3. The Work Load Forecast and Summary Report and the Manpower 
Planning Form (both forms found in Appendix B) are presented 
and each field is explained. Manpower is presented as a con- 
strained resource that must be managed. The last day of Phase I 
training requires each work center to submit a Manpower Plan- 
ning Form; turn-in procedures and use of the forms is explained. 
The forms are used to establish an initial ships database. 


4. Job scoping identifies the jobs for the ship's force work package 
(SFWP). The jobs for this package come from the ship's CSMP, 
the SARP, and other jobs the ship delineates. Job scoping breaks 
jobs down into steps accomplishable by one work center, called 
key operations (KEYOPS or KOP). The difference between origi- 
nating work center (OWC) and accomplishing work center (AWC) 
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is explained. (An OWC starts a job and is responsible to see the 
job through to completion. An AWC is responsible for a particular 
KEYOP, not the whole job.) 


5. Each field and the use of the FMS-RT Input Form, Work List 
Form, and Work List/Material Form are explained to the user. (All 
forms are found in Appendix B.) 


6. Phase I training concludes with the trainer explaining that FMS- 
RT is a dynamic system and changes can be made to all inputs. 
The trainer receives Manpower Planning Forms from each work 
center and arranges with the ship to send an initial list of jobs to 
him on FMS-RT Input Forms. These jobs are usually sent to the 
trainer about 30 days after Phase I training is complete. The 
trainer ends Phase I training by arranging with the command 
tentative dates for Phase II training, arranging a date for the ini- 
tial set of FMS-RT Input Forms to be mailed, and arranging an 
FMS-RT coordinator from within the ship to collect and dissemi- 
nate FMS-RT information and documentation. The trainer leaves 
about 10 copies of the FMS-RT Administrative Manual [Ref. 8] for 
command planning purposes to end Phase I training. The 
Administrative Manuals explain the overhaul process, the key 
players' responsibilities, staffing modifications, and other helpful 
information. 


Phase II training is 10 to 12 days in duration. However, Phase II 
training has been split into two parts provided that the government 
incurs no more costs than if it were conducted in one session. Phase II 
training is conducted aboard the ship. Phase II training consists of the 
following: 

1. Review of Phase I topics. 
2. The Scheduling Gantt Chart (found in Appendix B) is explained as 

a time management tool. Each field on the form and its purpose 


are explained to the users. 


3. Scheduling work is a broad topic. Scheduling entails the time it 
takes to do the activity (an estimate), time for quality assurance 
(QA), and a buffer or cushion. Scheduling must consider shipyard 
key events that may affect a job. The trainer explains how to build 
a buffer or cushion into jobs that must be done either prior to or 
in conjunction with a key event in the overhaul schedule. A 
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reminder is given to work center supervisors to pay attention to 
man-hours available to do industrial work. 


4. Balancing work involves even distribution of jobs based on the 
available man-hours from week to week. 


5. A second package of work on FMS-RT Input Forms is given to the 
trainer to add to the current ships database. 


6. Phase II training concludes with the trainer explaining that job 
estimates will be frozen at some point after the beginning of the 
overhaul. The trainer arranges a tentative date to bring the hard- 
ware and software to the ship and to train those people who will 
interface directly with the system. 
B. OPTIMAL CURRICULUM 

The optimal curriculum content was identified by three experts. 
Table 1 illustrates the questions asked each expert down the left side 
and the responses given by each expert across the top. The interviews 
were structured, so "N/A" indicates that the question is not applicable 
to that particular expert either because he had no experience in that 
area on which to base an answer or because the question was not rele- 


vant to that expert. 


C. USABILITY SURVEY RESULTS 

In our survey of users, we received 120 responses. Table 2 breaks 
down the respondents by their position in the chain of command and 
their experience in overhauls. Questions 1 and 2 in our survey ask for 
the user's position in the chain of command and how many major 
overhauls the user has gone through. In Table 2, positions are listed in 
the left column with the number of respondents and the number of 
major overhauls split into the mode response and the range of 


responses. 
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TABLE 2 
RESPONDENT PROFILE 


Position in # of # of Major Overhauls 

Chain of Command Respondents Mode Range 
Division Officer 36 1 (22) 1-2 
Chief Petty Officer (E7-E9) 22 2 (14) 1-6 
Work Center Supervisor 42 1 (30) 1-3 
FMS-RT Data Processor a 1 (12) N/A 
Others: 1 FMS-RT Coordinator 4* 2 (3) 1-2 
2 Department Head 3 2002) 1-2 

3 Commanding Officer 1 4 (1) N/A 

n = 120 


* 3 of 4 FMS-RT Coordinators are Department Heads; 1 was an E-8. 


“Mode” indicates the most frequently occurring value in a group 
of responses; the number in parentheses is the number of people who 
gave that response. For example, of the 120 surveys, 36 respondents 
were division officers. Twenty-two (22) of the 36 division officers 
responded that they had been through one major overhaul and the 
range of responses was one to two overhauls. N/A indicates that range 
is not applicable because all respondents gave the mode response. 

Questions 3 through 9 describe different usability components. 
The components have two descriptive scales for each question. Using 
one table per question, we list each scale and provide the percentage 
of respondents who selected each description; we put the actual num- 


ber of respondents in parentheses. The anchors range from one 


36 


extreme to an opposing extreme. The respondents are broken down 


by position across the top of the table with an overall percentage based 


on all respondents. 


TABLE 3 


DEGREE OF TRAINING 


Question 3: Degree of training provided on the FMS-RT and overhaul 


process: 

Division 

Officer 

Scale (n-36) 

Extremely 
complete 5.5%(2) | 0% (0) 
Quite complete 25% (9) | 22.7% (5) 
Slightly 
complete 50% (18) | 54.5%(12) 
Equally complete/ 
incomplete 5.5%(2) | 0% (0) 
Slightly 
incomplete 5.5% (2) | 0% (0) 
Quite incomplete 5.5% (2) | 13.6% (3) 
Extremely 
incomplete 3% (1)] 9.2% (2) 
Extremely 
superior 5.5% (2) | 4.5% (1) 
Quite superior 5.5% (2) | 0% (0) 
Slightly superior | 16.6%(6) | 9.0% (2) 


Equally superior/ 
inferior 33.3%(12)| 59.5%(13) 
Slightly inferior | 25% (9)|18% (4) 


Quite inferior 14.190(5) | 4.5% (1) 
Extremely 
inferior 0% (0)| 4.5% (1) 


4.7% (2) 
23.7%(10) 


50% (21) 


4.7% (2) 


2.4% (1) 
14.5% (6) 


0% (0) 
0% (0) 
4.7% (2) 
28.8%(12) 
42.8% (18) 
9.5% (4) 
9.5% (4) 


4.7% (2) 


37 


0% (0) 
12.5%(1) 


62.5%(5) 
12.5%(1) 


0% (0) 
12.5% (1) 





0% (0) 


0% (0) 
0% (0) 
37.5% (3) 


50% (4) 
12.5%(1) 
0% (0) 


0% (0) 


Overall 
(n-120) 


3.3% (4) 
22.59% (27) 


54.2% (65) 
4.2% (5) 


3.3% (4) 
1096 (12) 


2.5% (3) 
2.5% (3) 
4.2% (5) 
215790026) 
44.2% (53) 
15.8%(19) 
9.1%(11) 


2.5% (3) 


Question 4: 


Scale 





Extremely 
sufficient 
Quite sufficient 
Slightly 
sufficient 
Equally 
sufficient / 
insufficient 
Slightly 
insufficient 
Quite insufficient 
Extremely 
insufficient 
Extremely 
complete 
Quite complete 
Slightly complete 
Equally 
complete / 
incomplete 
Slightly 
incomplete 
Quite incomplete 
Extremely 
incomplete 


UNDERSTANDING OBJECTIVES 


Division 
Officer 
(n-36) 


11.1% 
16.7% 
38.8% 


(4) 
(6) 
(6) 


16.7%(14) 


16.7% (6) 
0% (0) 
0% (0) 


TABLE 4 


0% (0) 
4.5% (1) 
9% (2) 
46% (10) 
27% (6) 
13.5% (3) 
O% (O) 
18% (4) 
32.2% (7) 
27.2% (6) 
13.6% (8) 
0% (O) 
9% (2) 
4.5% (0) 


2.4% (1) 
4.7% (2) 


4.7% (2) 


28.8%(12) 


9.5% (4) 
26.2%(1 1) 


23.7%(10) 
2.4% (1) 
21.4% (9) 
23.3%(10) 
38.8%(16) 


4.7% (2) 
4.7% (2) 


4.7% (2) 
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0% (0) 
0% (0) 


8.4%(1) 


33.3% (4) 


33.3% (4) 
25% (3) 


0% (0) 
0% (0) 
25% (3) 
8.4%(1) 
58.2%(7) 


0% 
0% 


(O) 
(O) 


8.4% 1) 


O96 (0) 
O96 (0) 


0% (0) 


12.596(1) 


12.596(1) 
81% (6) 





0% (0) 


O96 (0) 
20% 12) 
50% (4) 


25% (2) 


0% (0) 
O96 (0) 


0% (0) 


User's understanding of the objectives of the FMS-RT: 


Overall 
(n-120) 


0.8% (1) 
5% (6) 


6.7% (8) 


30% (36) 


17.5%(21) 
24.1%(29) 


15.9%(19) 
7.5% (9) 


22.9% (27) 
22.5%(27) 


35% (42) 


6.7% (8) 
3.3% (4) 


2.5% 0) 


Question 5: 


Scale 


Extremely 


positive 


Quite positive 
Slightly positive ¡25% (9) 
Equally positive/ 


negative 


Slightly negative {19.5% (7) 
Quite negative 


Extremely 
negative 


Extremely 
sufficient 
Quite sufficient 


Slightly 


sufficient 


Equally 


sufficient / 
insufficient 


Slightly 


insufficient 


TABLE 5 
UNDERSTANDING OBJECTIVES 


User's feeling of participation in the overhaul manage- 
ment process by using this system: 


Division 


Officer Overall 
(n-36) (n-120) 





11.1% (4) 
11.1% (4) 


0% (0) | 8.4%(1) 4.2% (5) 
0% (0) | 9.5% (4) | 33.396(4) [100% 16.6% (20) 
9% (2) |31% (13) |33.3%(4) 23.3% (28) 
33.3%(12)| 37% (8) | 50% (21) [25% (3) 36.6% (44) 
22.5% (5) | 2.4% (2) | 0% (0) 11% (13) 


096 (0)| 13.5% (3) | 2.4% (1) | 0% (0) 3.3% (4) 





0%  (0)|18% (4) | 4.796 (2)| O96 (0O) 15% (6) 





5.5% (2)| 9.2% (2) | 2.4% (1) | 0% (0) | 0% (0) 4.2% (5) 
096  (0)| 40.996 (9 | 0% (0)| 0% (0) | 0% (0) 7.596 (9) 


11.196 (4)| 31.896 (7) | 0% (0) |16.6%(2) | 0% (0) | 10.9%(13) 


11.196 (4)| 18.196 (4) | 48% (2) [58.4% (7) |12.5%(1) | 15% (18) 


33.3%(12)| 0% (0) | 21.4% (9) [25% (3) |62.5%(5) | 24.1% (29) 


Quite insufficient [22.3% (8)| 0% (0) | 71.4%(30) | 0% (0) [25% (2) | 33.3%(40) 


Extremely 
insufficient 


16.7% (6)| 0%  (0)| 0% (0) | 8.4%(1) | 0% (0) 5% (6) 
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TABLE 6 
ATTITUDE TOWARD FMS-RT 


Question 6: Attitude toward using FMS-RT: 


FMS-RT 
Division Data 


Officer Processor 









Scale (n-36) (n-12) 

Extremely 

cooperative 5.5% (2)| 4.5% (1D | 0% (0)] 8.3%(1) 
Quite cooperative {25%  (9)| 13.596 (3) | 14.396 (6) | 16.8%(2) 
Slightly 

cooperative 50% (18 996 (2)|1996 (8) [25% (3) 
Equally 

cooperative/ 

belligerent 8.3% (3)|50% (11) |38% (16) [25% (3) 
Slightly 

belligerent 2.896 (1)| 18.596 (4) | 14.396 (6) | 8.396(1) 
Quite belligerent 5.5% 0% (0) | 7.2% (3) | 8.3%(1) 
Extremely 

belligerent 2.9% (1)| 4.5% (1) | 7.2% (3) | 8.3%(1) 
Extremely 

negative O96 (O0) O96 (0)| 0% (0) | 0% (0) 
Quite negative 0%  (0)1 13.5% (3) | 7.2% (3) | 0% (O) 


Slightly negative |11.196 (4)| 22.796 (5) | 14.396 (6) | 0% (0) 
Equally negative/ 


positive 22.2% (8)| 22.7% (5) | 47.3%(18) |25% (3) 
Slightly positive |39% (14)|27.6% (6) [21% (9) |33.3%(4) 
Quite positive 22.296 (8)| 13.596 (3) | 5.5% (4) | 33.396(4) 
Extremely 

positive 5.5% (2)| 0% (0) | 4.7% (2) | 8.4%(1) 
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0% (0) 
12.5% (1) 


0% (0) 


62.5% (5) 


0% (0) 
25% (2) 





0% (0) 


0% (0) 
0% (0) 
25% (2) 


25% (2) 
37.5% (3) 
12.5%(1) 


O96 (0) 


Overall 
(n-120) 


3.3% (4) 
17.5% (21) 


25.8% (31) 


31.6% (38) 


10% (12) 
6.8% (8) 


5% (6) 
O96 (0) 
5% (6) 
14.1%(17) 
30% (36) 
30% (36) 
16.7% (20) 


4.2% (5) 


TABLE 7 


RELIABILITY OF OUTPUT 


Question 7: Reliability of output information: 


Scale 





Extremely high 
Quite high 
Slightly high 
Equally high/low 
Slightly low 
Quite low 
Extremely low 
Extremely 
complete 
Quite complete 
Slightly complete 


Equally complete/ 


incomplete 
Slightly 

incomplete 
Quite incomplete 
Extremely 

incomplete 


Division 
Officer 
(n-36) 


28% 
28% 
11.1% (4) 


33.3% (12) 
11.1% (4) 
27.8%(10) 
11.1% (4) 


16.7% (6) 
8.3% (3) 
90% (18) 
16.7% (6) 


9.9% (2) 
0% (0) 


2.8% (1) 


CPO 
(n-22) 


0% (0) 
4.5% (1) 
22.7% (5) 
36.4% (8) 
0% (0) 
36.4% (8) 
0% (0) 


18.2% (4) 
13.6% (3) 
27.5% (6) 
22.7% (5) 


996 (2) 
4.5% (1) 


4.5% (1) 


WCS 
(n-42) 


4.7% (2) 
21.4%(16) 
37.2% (9) 
26.2%(11) 

0% (0) 

7.1% (3) 

2.4% (1) 


7.1% (3) 
4.7% (2) 
21.4% (9) 
16.6%(18) 


4.7% (2) 
43.1% (7) 


2.4% (1) 
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FMS-RT 


Data 


(n-12) 


0% (0) 
16.7%(2) 
16.790(2) 
2596 (3) 
33.396(4) 
8.3%(1) 
0% (0) 


0% (0) 
33.3% (4) 
33.3% (4) 
33.4% (4) 


0% 
0% 


(0) 
(0) 


0% (0) 


Processor| Others 


(n-8) 





0% (0) 
37.5% (3) 
O96 (0) 


37.5%(3) 


0% 
25% 


(O) 
(2) 


0% (0) 


Overall 
(n-120) 


2.5% (3) 
16.7% (20) 
20% (24) 
31.6% (38) 
6.6% (8) 
18.3% (22) 
4.3% (5) 
10.8%(13) 
12.6%(15) 
30.8% (37) 
30% (36) 


5% (6) 
8.3%(10) 


2.5% (3) 


Question 8: 


Scale 


Extremely 
relevant 
Quite relevant 
Slightly relevant 
Equally relevant/ 
irrelevant 
Slightly 
irrelevant 
Quite irrelevant 
Extremely 
irrelevant 
Extremely useiul 
Quite useful 
Slightly useful 
Equally useful/ 
useless 
Slightly useless 
Quite useless 
Extremely useless 


TABLE 8 


RELEVANCY OF OUTPUT 


Division 
Officer 
(n-36) 


0% (0) 
11.1% (4) 
19.4% (7) 


98.5% (21) 


0% 
5.5% 


(0) 
(2) 


(2) 
(O) 
(5) 
(6) 


5.5% 

0% 
14.1% 
16.6% 


50% (18) 
8.3% (3) 
5.5% (2) 
9.500 (2) 


0% (0) 
0% (O) 
27.2% (6) 


36.6% (8) 


13.6% (3) 
996 (2) 


13.696 (3) 
0% (O) 
996 (2) 
27.496 (6) 


27.496 (6) 
13.6% (3) 
13.6% (3) 
99oo 12) 


2.496 (1) 
1996 (8) 
26.3%(11) 


26.2%(11) 


21.4% (9) 
4.7% (2) 


O96 (O) 
4.7% (2 
19% (8) 
23.8%(10) 


23.8% (10) 
23.9%(10) 
2.4% (1) 
2.4% (1) 
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0% (0) 
8.3%(1) 
41.7%(5) 


41.7%(5) 


0% (0) 
8.3%(1) 


0% (0) 
0% (O 
16.6%(2) 
33.3% (4) 


25.2%(3) 
16.6% (2) 
8.3%(1) 
0% (0) 


Processor] Others 


0% (0) 
12.5%(1) 
12.5%(1) 


37.5% (3) 


12.5%(1) 
25% (2) 


0% (0) 
0% (0) 
12.5% (1) 
12.596(1) 





37.596(3) 
12.5%(1) 
25% (2) 

0% (0) 


Relevancy of output information to overhaul planning: 


Overall 
(n-120) 


0.8% (1) 
11.5% (14) 
25% (30) 
40% (48) 
11% (13) 

7.5% (9) 


4.2% (5) 
1.6% (2) 
15% (18) 
22.999122] 


33.3% (40) 
15.9%(19) 
7.599819) 
4.296 (5) 


TABLE 9 
OUTPUT ACCURACY 


Question 9: Accuracy of output information: 





Division 
Officer Overall 
Scale (n-36) (n-120) 
Extremely 
accurate 11.196 (4)] O96 (O) | 2.4% (1) | 0% (0) 4.3% (5) 
Quite accurate 27.8% (10) | 36.4% (8) 7.1% (3) | 8.3%(1) 18.3% (22) 
Slightly accurate [11.1% (4)} 0% (0)| O% (0) [33.3%(4) 6.6% (8) 
Equally accurate/ 
inaccurate 33.3%(12)| 36.4% (8) | 26.2%(11) [25% (3) 31.696(38) 
Slightly 
inaccurate 11.196 (4)| 22.796 (5) | 37.296 (9) |16.796(2) 2096 (24) 
Quite inaccurate 2.8% (1)| 4.5% (1) | 21.4%(16) | 16.7%(2) 16.7%(20) 
Extremely 
inaccurate 2.896 (1)] O96 (0) | 4.7% (2) | 0% (0) 2.5% (3) 
Extremely certain |16.7% (6)| 18.2% (4 7.1% (3 | O% (0) | O% (0) | 10.8%(13) 
Quite certain 8.396 (3)| 13.696 (3) | 4.796 (2) | 33.396(4) | 37.596(3) | 12.6%(15) 


Slightly certain [5096 (18)| 27.596 (6) | 21.496 (9) [33.396(4) | O96 (0) | 30.8%(37) 
Equally certain/ 

uncertain 16.7% (6)|22.7% (5) | 16.6%(18) | 33.4%(4) |37.5%(3) | 30% (36) 
Slightly uncertain| 5.5% (2)| 9% (2) | 4.7% (2) | 0% (0) | 0% (0) 5% (6) 
Quite uncertain 0%  (0)| 4.5% (1) | 43.1% (7) | 0% (0) [25% (2) 8.3% (10) 
Extremely 

uncertain 2.8% (1)| 4.5% (1) | 2.4% (1) | 0% (0) | 0% (0) 2.5% (3) 
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We investigated the number of people involved in controlling data 
entry, or what we call “communicating changes.” The users on our 


survey indicated the following (n=120): 


Too many 1.67% DESEN 
Just right 80% (n-96) 
Too few 18.33% [m o) 


As you Can see, the vast majority of users feel that just enough people 
are involved in controlling data entry. The number of changes by a 
work center to the database on a per weekly basis is 8.5 changes per 
week. The number of changes per week was ascertained by averaging 
the number of changes indicated by work center supervisors. The total 
number of changes to the database can be obtained by multiplying the 
number of work centers by the 8.5 change per work center in a week. 

Questions 12 and 13 in our survey are planned enhancements for 
the FMS-RT system. We surveyed the users to see if these two 
enhancements are desired. First, would the user like FMS-RT to be 
interactive? The term “interactive” was explained to the user by 
making the user aware that the SNAP I and II systems currently on all 
CRUDES-type ships is an interactive system, interactive meaning that 
the person changing the data makes the changes himself at the termi- 
nal. The users' responses were an overwhelming “yes! “Yes” 
responses were 108 to 12 “no” responses. 

Second, are the users interested in seeing data from other ships 


in their class? The users responded as follows: 
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I [the user] see(s) the need 
to look at other ships' data 92% (1262) 


I [the user] see(s) no need 
to look at other ships' data 48% (n=62) 


Questions 14 through 16 were added to verify the researcher's 
observations. First, the researcher observed that the generated reports 
seemed only useful to work center supervisors. The higher levels in 
the chain of command found the report formats vague and not very 
useful. Table 10 illustrates how each level in the chain of command 
responded to the question “Are the reports generated in a useful 
format?” 

Second, the researcher observed during training that the reports 
were presented. However, the training was an explanation of the 
fields, rather than how to interpret the output and apply the results to 
the overhaul management process. The survey participants were asked 
whether they felt their training taught them how to use the output 
reports as a tool to manage the overhaul process. 

Finally, the researcher observed that nearly all users had a manual 
tracking system in addition to the computer-based FMS-RT system. 
Table 10 shows how many of each level used a manual tracking system. 

The most important perception to get from the user is the overall 
usability of the FMS-RT system. Table 11 illustrates how the different 


levels in the chain of command feel about overall usability. 
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TABLE 11 
OVERALL USABILITY 







FMS-RT 
Data 
Processor 
(n-12) 










Division 
Officer 
(n-36) 






Overall 
Usability 






Overall 
(n-120) 


Extremely useful 1 O O 0 O 1 
Quite useful 6 4 20 4 D 37 
Better than 

nothing 2l 9 12 5 3 50 
More of a problem 

than a useful 

tool 8 9 10 3 2 o 


The last part of our survey asked the user to list any problems or 
enhancements. Appendix D is a listing of the weaknesses in FMS-RT 
and possible enhancements provided by the users. The numbers in 
parentheses after the brief explanation indicate the numbers of 


respondents with a similar problem or a similar enhancement. 
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VI. ANALYSIS 


A. TRAINING CONTENT 
In this section, the researcher will discuss the compatibility 
between the current training curriculum and the optimal curriculum 
as defined by the experts. The experts provided a list of topics they 
found important in an optimal training curriculum. It was found that 
the topics in the current FMS-RT training program are the same as 
those in the ideal curriculum with the exception of three subject 
areas: project management training, advanced project management 
estimating techniques, and the assignment of responsibilities to 
implement the program. 
1. Project Management 
In this section, both project management and advanced esti- 
mating techniques will be covered, since both relate to project man- 
agement. In a question to all the experts, the researcher asked “How 
important is project management training in a curriculum?” All three 
experts agreed that project management training is essential to con- 
ducting a successful overhaul. Project management is distinguished 
from production management primarily by the non-repetitive nature of 
the work; a project is usually a one-time effort. The management of 
projects is more complicated than the management of a production 
line due to the following characteristics, generally typical of project 


management to some degree. The duration of a project can last weeks, 
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months, or even years. During such a long period many changes may 
occur, most of which are difficult to predict. Such changes may have a 
significant impact on project costs, technology, and human resource 
allocation. The longer the duration of the project, the more uncertain 
are the times and costs. The project can be complex in nature, 
involving many interrelated activities and participants from both 
within the organization and outside the organization. Delays in 
completion time may be very costly. Penalties for delays may amount to 
thousands of dollars per day. Completing projects late may result in 
lost opportunities and ill will. Projects are sequential in nature. Some 
activities cannot start until others are completed. Projects are typically 
a unique undertaking, something that has not been encountered fre- 
quently. Thus, the experts thought that, given the complex nature of 
overhauls and integration problems created between contractor work 
and ships force work, project management should be an essential part 
of the training curriculum. Additionally, given the time constraint of 
when the overhaul will end, the experts thought that better activity 
duration estimates would allow better overhaul management. There- 
fore, the experts felt advanced project management estimating tech- 
niques were an equally important part of project management training. 

The follow-on question was “At what level in the chain of 
command should this training be given?” The experts' responses var- 
ied from Chief Petty Officers to department heads. One expert felt that 
it should be taught at the Chief Petty Officer level, but all three agreed 
that it should be taught at the division officer and department head 
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level. The experts' rationale for including project management train- 
ing at these levels was that management was basically the job of the 
division officer. However, the experts clarified this by stating that 
project management is complex and they felt more experience in ship 
repair is needed to understand the complex interrelationships of an 
overhaul. For this reason, the experts felt CPOs and department heads 
could better understand project management and utilize project man- 
agement tools during the overhaul. 
2. Assignment of Responsibilities 

During an overhaul, personnel are given additional manage- 
ment tasks specific to the overhaul in addition to their regular duties. 
The FMS-RT Administrative Manual explains the responsibilities of 
certain key overhaul personnel. However, these responsibilities are 
general in nature. According to the experts, they need to be more 
clearly defined by the ship’s commanding officer. People need to know 
what they are expected to do during the overhaul, an then they need 
to know how to accomplish this tasking. The experts feel that this 
important framework needs to be laid prior to any other training, and 
that not enough effort is being expended in this regard. The experts 
also believe that the curriculum must ensure that individual unit plan- 
ning and training cover the “big picture” items prior to FMS-RT 
training. The most important thing is to have the overhaul manage- 
ment assignments made and their responsibilities clearly delineated 


in advance. 
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In summary, according to the experts, the FMS-RT training 
curriculum contains the necessary content with three exceptions: 
project management training, advanced estimating techniques, and 
assignment of responsibilities. The experts recommend that these 


additional topics be incorporated into the current FMS-RT curriculum. 


B. USABILITY 
In Table 11, 50 of 120 respondents said that FMS-RT was better 
than nothing. Only one respondent found it extremely useful. What 
causes the users to evaluate the FMS-RT system as they have? 
The following problems emerged in analyzing survey responses 
and observing users: 
1. CPO Participation 
CPOs do not feel that their participation in the overhaul man- 
agement process is either sufficient or positive. This is evident in 
Table 5, where about 60 percent of the respondents are in the insuffi- 
cient and negative ranges of the sales. 
2. User Attitude 
The attitude of the users is marginally cooperative. Table 6 
indicates that for all levels there is only a slightly positive and slightly 
cooperative attitude toward using FMS-RT. Combining the results 
found in Table 6, with the 10 respondents in Appendix D who 
requested an initial SFWP created for the ship, as SFOMS did previ- 
ously, indicates an FMS-RT system acceptance problem. Some addi- 


tional responses from the survey participants may explain why 
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acceptance is a critical issue. The following quotes indicate that the 
FMS-RT system is not as uselul as we might like it to be: 


FMS-RT is all right for early in the overhaul, but useless in the 
latter phases. It does not have enough detailed information on the 
status of jobs and the pretty graphs tell me very little about the 
problems or about what caused them. [CDR Loeffler, Ref. 11] 


The reports are not in a useful format and the system is too com- 
plex. I own my own personal computer and could build a better 
and easier to use system in six months. [Survey from a CPO in USS 
Underwood (FFG-36)] 


3. Output 
Although the respondents say that the output is reliable and 
accurate in Tables 7 and 9, the researcher observed that the users find 
minor inaccuracies because all their changes are not yet entered and 
reflected in their output reports. Additionally, they are uncertain as to 
what the reports should tell them. In Appendix D, 17 respondents 
requested more training on how to use output forms in managing the 
overhaul. Table 10 indicates that an overwhelming number of respon- 
dents feel that the training did not teach them how to use the output 
forms in managing the overhaul process. 
4. Reports 
Table 10 indicates that reports are not in a useful format for 
upper levels in the chain of command. This point was echoed by CDR 
Loeffler in the quote above and by 15 respondents in Appendix D. 
5. Interactive System 
108 of 120 respondents want an interactive system. The 
researcher observed that work center supervisors wanted to make 


changes to their own data at their (the work center supervisors? 
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convenience. The work center supervisor want their autonomy when 
dealing with the FMS-RT system. Currently, the supervisors feel that 
they have to follow a command-directed schedule for report submis- 
sion. The researcher believes the data quality would increase if the 
FMS-RT system were integrated with the SNAP I and II systems. The 
work center supervisors would then be responsible for updating their 
part of the data. This solution also would eliminate the users' feeling 
that output was only relatively reliable and accurate. 
6. Training 

Appendix D, a summary of the respondents' open-ended 
comments, shows training is incomplete. However, the researcher in 
investigating the problem in more detail called the commands that 
responded with this point for further clarification. The problem was 
found to be more a transition problem than a training problem. The 
ships in overhaul prior to January 1988 had SFOMS and had to be 
converted over to the FMS-RT system. The transition was difficult 
because the change was abrupt; the training was short, fast-passed, 
and specific to FMS-RT; and the software had problems that had not 
been found in testing. Abrupt means that SFOMS was gone on 1 Jan- 
uary, and FMS-RT was put in and training held with little notice to the 
ship. The training highlighted the similarities and differences between 
the two systems and focused on the differences in the output reports. 
The software problems created a major problem in getting reports out 
of the system and created a distrust of the FMS-RT system. The 


researcher concluded that the crew was overloaded in the transition 
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training. The two-phase training program given currently should pre- 
vent the recurrence ol this problem. 
7. Manual Tracking System 
Table 10 indicated that many respondents use a manual 
tracking system in addition to the FMS-RT system. This indicates that 
FMS-RT does not provide all the information users want. Appendix D 
illustrates that respondents want different reports created and more 
fields added to existing reports for such information as detailed 
remarks and percentage of job completed. 
8. Material Tracking System 
The researcher observed that very few commands used the 
material tracking system in FMS-RT. The system is manual in nature 
and requires constant updates to be useful. The researcher observed 
that during overhaul, the Supply Department loses one or two store- 
keepers to the SOAP team, and receives the most requisitions to han- 
dle. Therefore, they have the most work to do with the fewest people 
to do the work. This is one reason that the additional material track- 
ing provided by FMS-RT is not used. Also, the SNAP system has 
tracking capability, but SNAP does not generate reports and requires 
the same constant updating as FMS-RT. Thus, we can understand why, 
12 people requested that material tracking be totally automated (see 
Appendix D). 
9. Software Problems 
In Appendix D, we see many problems with software that 


should have been found during acceptance testing. Problems with form 
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feeds, error messages, date printouts on reports, and compartment 
numbering on reports should have been caught during the testing 
phases. Other problems, like screen colors and consistency in menu 
selections, indicate little user involvement in design and lack of con- 
cern over usability issues by the developer. 

Generally, the FMS-RT system is complex and difficult to use. 
Stahl's research on usability illustrates that testing for usability can 
head off disaster [Ref. 15:p. 83]. Stahl indicates that more and more 
companies are conducting usability tests in addition to testing for 
bugs. He suggests getting users involved in all phases of software 
development. Stahl recommends the use of prototyping, videotaping of 
end-user learning, and objective and subjective testing. Stahl says that 
it is not enough for software to perform correctly, it must be usable by 
the end user. The end users must be satisfied. 

In the case of the FMS-RT system, the data reveal that not all 
end users are satisfied. The reports are not in a useful format for some 
levels of the chain of command. The software was sent to the fleet 
with bugs that should have been found during testing. Task perfor- 
mance time, such as for inputting data, is hampered by poor screen 
color selection. System learning time is slow due to documentation 
that is written for a level above the normal user. Also, the documenta- 
tion is missing things like dip-switch settings for the printer. 

In summary, it appears that the FMS-RT was moved from 
development to fleet implementation too quickly. The software 


needed better testing and more usability features. The documentation 
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needed to be written for the general user, as mentioned earlier. The 
reports should have been generated by the users through the use of 
existing report generators and more reports were needed than are 
currently provided. The researcher. having seen the system in opera- 
tion and having reviewed the users' comments, believes that the life 
expectancy for FMS-RT will be short unless a new system is developed 
with user involvement in all phases of development. The system 
should conform to DOD-STD-2167A on Defense System Software 
Development [Ref. 16] and MIL-STD-1472C Notice 3 on Human Engi- 
neering Design Criteria for Military Systems, Equipment, and Facilities 


[Ref. 17]. 


96 


VII. RECOMMENDATIONS AND CONCLUSION 


The premise of this thesis is that, with a well-developed training 
curriculum and usable software, we will get increased performance 
during an overhaul. The researcher concludes that we have a well- 
developed training curriculum but we have a system that is difficult to 
use. The FMS-RT system frustrates some end users. 

In response to the experts recommended curriculum changes, 
the researcher believes that project management training can be con- 
ducted in one of three ways. First, it can be incorporated into the cur- 
riculum at Surface Warfare Officers School. The training at the existing 
schools are at the division officer level and the department head level. 
Therefore, both levels can be taught project management techniques 
and the curriculum content (advanced or introductory training) can be 
controlled. Controlling the content can allow the training curriculum 
to include the advance estimating techniques the experts mentioned, 
like weighted averages and PERT estimating (a form of weighted aver- 
age that uses three estimates— an optimistic, a pessimistic, and a nor- 
mal estimate), until the advanced training is conducted. Second, this 
training can be incorporated into the FMS-RT training and be given 
specifically to all three levels (CPOs to department heads) just prior to 
overhaul. The last way is to teach project management at Surface War- 


fare School and during FMS-RT training. 
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Since a great deal of what officers are supposed to do is manage, 
the placement of project management training would reap the most 
benefits if it were taught at an introductory level in the Surface War- 
fare Officer Basic Course, and at a more advanced level in the Surface 
Warfare Officer Department Head Course. The officers would then be 
able to better manage overhauls, train subordinates in these tech- 
niques, and ideally be better overall managers. A converse argument is 
to teach project management during FMS-RT training aboard the ship. 
One could argue that it would be fresh in the manager's mind if it 
were taught to him just prior to overhaul, and all three levels could 
participate. The ideal would be to do both— teach project management 
at Surface Warfare School and during FMS-RT training. Further analy- 
sis would be needed to cost justify incorporation of any of these com- 
ponents into any curriculum. 

However, if we could not teach this topic in both curricula, then 
the researcher believes that teaching it in Surface Warfare School at 
both levels is the best approach. The rationale behind this choice is 
that project management techniques apply to more than just over- 
hauls. The researcher believes that these skills can be used in small 
availabilities when FMS-RT is not provided. Also, these skills are fre- 
quently required by officers in follow-on tours. Project management 
bilets are a large portion of senior officer billets. Also, all three 
experts agreed that, at a minimum, division officers and department 


heads need this type of training. For these reasons, the researcher 
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recommends placing project management training in Surface Warfare 
School. 

The training curriculum can be improved by putting more 
emphasis on how to utilize the output reports as a project manage- 
ment tool. Also, the training curriculum must incorporate project 
management training for middle-level managers (CPOs to department 
heads). The training content should include advanced estimating 
techniques, PERT and CPM techniques, and methods for interpreting 
computer-generated output. 

The software is complex and difficult to use. The end users are 
not satisfied. The system can be improved by implementing the 
enhancements requested in Appendix D and also by correcting the 
problems presented. The enhancements include more forms, addi- 
tional fields in existing forms, automation of the material tracking 
function, and the creation of an interactive environment. The prob- 
lems include form-feed problems, compartment numbering errors, 
date listing problems on Gantt charts, and error message disappear- 
ance before the user can read the message. 

In the process of this research, issues for further evaluation came 
to light. The following is a list of follow-on research that is 


recommended. 


e Conduct a cost-benefit analysis to decide whether to implement 
the proposed enhancements and corrections of the identified 
problems as a revision of the software or to develop a new, more 
usable system. 


e Conduct specific research with users to determine the type of 
reports needed and their formats. This would be extremely 
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helpful to the future developer of either the enhancements or a 
new system. 


e Evaluate the training methods and the trainer. This research eval- 
uated the training curriculum but not the training methods or the 
trainer. Evaluation in these areas of training is warranted. The 
researcher believes better-quality training can be obtained by 


using videotapes and tutorials. Also, having participated in the 
current training, the researcher found it dull and uninspiring. 

e Investigate the possibility of having a contractor do the total over- 
haul while the ships force is sent to other ships of the same class 
for training or to advanced schools. The Canadian Navy overhauls 
ships this way and is saying it gets a higher-quality overhaul at a 
lower coast. Also, the crew is better trained because crew mem- 
bers do not forget things about their systems during the long 
overhaul process. The crew is training while the contractor is 
working. 


e Investigate implementation of an overhaul tracking system on the 
SNAP I and II systems. 


° Investigate methods of automating the material tracking system of 
FMS-RT. How can we link the ship to the information and status 
of requisitions in the supply system? 


The Navy relies on computer-based systems like FMS-RT to 
improve its performance. However, the Navy has not provided the 
software developers access to the end users. This has led to complex 
systems that are difficult for the user to understand and use. To main- 
tain our advantage strategically and to increase performance opera- 
tionally, we must develop systems that are not only correct but that 
can be used. Usability is more than an issue of the 1980s, it is an issue 
that makes or breaks our systems. It is imperative that we concentrate 
on identifying those areas where automation will permit personnel to 
perform to their maximum potential, then develop these systems for 
the people to use, always considering their knowledge level and capa- 


bilities. With these efforts, we can develop specific training programs 


60 


to obtain maximum performance from both the people and the com- 


puter system. 
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APPENDIX A 


LIST OF ABBREVIATIONS 


NAVY COMMANDS AND OTHER ORGANIZATIONS 


CMSD 


NAVSEA 
OMB 


PERA 


PERA (ASC) 


PERA (CSS) 

PERAN(CY) 

PERA (CRUDES) 
SEAADSA 


SUPSHIPS 


TYCOM 
COMNAVSURFLANT 


COMNAVSURFPAC 


Continental Maritime of San Diego 
(contractor) 


Naval Sea Systems Command 


Office of Management and Budgeting 
(Executive Office of the President 


Planning and Engineering for Repairs and 
Alterations 


PERA for Amphibious and Support Craft 
(Norfolk) 


PERA for Combat Support Craft (Bremerton) 
PERA for Aircraft Carriers (San Francisco) 
PERA for Surface Combatants (Philadelphia) 


Naval Sea Systems Command Automated 
Data Support Activity 


Supervisor of Shipbuilding, Conversion and 
Repair 


Type Commander 


Commander Naval Surface Force Atlantic 
Fleet 


Commander Naval Surface Force Pacific 
Fleet 
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TYPES OF OVERHAULS/AVAILABILITIES 


COH 
DPMA 
DSRA 
EDPMA 


EDSRA 


ROH 


OTHER ABBREVIATIONS 


AWC 

CPM 
CSMP 
FMP 
FMS-RT 
ILO 

IMA 
INFONET 
INSURV 
KEYOPS/KOP/KEY OP 
MIS 
MOTU 
OVHL 
OWC 
PERT 


Complex Overhaul 

Drydocked Phased Maintenance Availability 
Drydocked Scheduled Restricted Availability 
Extended Drydocked Phased Maintenance 
Availability 

Extended Drydocked Scheduled Restricted 
Availability 


Regular Overhaul 


Accomplishing Work Center 
Critical Path Method 

Current Ship's Maintenance Project 
Fleet Modernization Program 

Fleet Management System-Real Time 
Integrated Logistics Overhaul 
Intermediate Maintenance Activity 
Information Network 

Board of Inspection and Survey 

Key Operation 

Management Information System 
Mobile Training Unit (an IMA) 
Overhaul 

Originating Work Center 


Program Evaluation Review Technique 
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POT&I 
QA 
SARP 
SFOMS 
SFWP 
SIMA 


SITREP 
SOAP 
SS/SC 


3M 


Pre-Overhaul Test and Inspection 

Quality Assurance 

Ship's Alteration and Repair Package 
Ship’s Force Overhaul Management System 
Ship’s Force Work Package 


Shore Intermediate Maintenance Activity 
(an IMA) 


Situation Report (Commanding Officer's) 
Supply Overhaul Assistance Program 


Scheduled Start/Scheduled Completion 
(scheduling) 


Material, Maintenance, Management 
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APPENDIX B 
FMS-RT INPUT AND OUTPUT FORMS 


A. FMS-RT INPUT FORMS 
The FMS-RT has been developed with the idea that input forms 
and paperwork are to be kept to a minimum; therefore, there are 
relatively few input forms. All forms are graphically produced on the 
computer screen and can be utilized for all input functions with the 
exception of the Material Input and OPNAV 4790/2K forms. 
1. Forms 
e OPNAV 4790/2K 
e FMS-RT Input Form (Optional page 2) 
e FMS-RT Key Event Data Form 
e FMS-RT Material Request Form 
e NAVSUP 1250-1 
e DD 1348 
e FMS-RT Manpower Planning Form 
e FMS-RT Material Status Form 
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2. OPNAV 4790/2K Form 

The OPNAV 4790/2K/20/2R Form is the input form for the 
ships 3-M System. Whether a ship is in or out of an industrial avail- 
ability, a 4790/2K/2Q/2R must be submitted for each job that must be 
accomplished. Information provided via this form (JCN, noun name, 
EIC, CSMP summary, etc.) is required in the FMS-RT data file. When 
the work is accomplished, the 4790/2K is sent to the CSMP to update 
the ship's maintenance records. 

If work cannot be accomplished within 30 days or is to be 
deferred until an industrial availability period, that work must be 
entered, via the OPNAV 4790/2K form, into the ships CSMP. Concep- 
tually, FMS-RT is to interface with the 3-M System (CSMP) to reduce 
documentation. Therefore, job information (JCN, noun name, EIC, 
CSMP summary, etc.) contained on either of these forms is used in the 
FMS-RT data file. The OPNAV 4790/2K/20/2R Form is the primary 


form used to load repair (industrial) work into the FMS-RT data file. 
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OPNAV 4790/2K 
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3. FMS-RT Input Form 

A FMS-RT Input Form must be submitted for each job to 
enter it into FMS-RT. After jobs have been scoped, initial job and 
KEYOP data are sent to the FMS-RT master file on this form. It can 
also be used to delete or change existing job data (Section I) and 
KEYOP data (Section II) and to add additional (new) KEYOPS to exist- 
ing jobs. Workspace is provided (COMMENTS) for pertinent com- 
ments concerning the job or KEYOP. 
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LEGEND 


1. 
2. 
3. 
4 


S. 


Ship's Name and Hull Number 
Unit Identification Code (UIC) 
Job Control Number (JCN) 
Allowable Parts Lists/Allowable 
Equipage List (APL), (AEL) 


Equipment Identification Code 
(EIC) 


SECTION I— JOB INFO 


6. 
7. 
8. 
g 


10. 
11. 


F2. 


Action Code Noun 

Job Name 

Priority 

Job Level Key Event Number 
Mission Essential Number 
genas that will amplify the 
jo 


Screening Action by TYCOM 
(assignments of responsibilities) 


SECTION II— KEYOP INFO 


13. 
14. 
15. 


Action Code 
Percentage of Total Job 
KEYOP Level Key Event 





16. 
17. 


18. 


19; 
20. 
21. 


22. 


25 


24. 


26 — — — — — 


KEYOP “Critical” Flag* 


Accomplishing Work Center 
(AWC) 


KEYOP Number Assigned to 
Each Entry 


Description of KEYOP 
Compartment Number 


Scheduled Start Week (SS) and 
Scheduled Completion Week (SC) 


Man-Hours Estimated (in whole 
hours) to Accomplish the KEYOP 


Quality Control Requirement 
(optional) 


For Future Use 


COMMENTS SECTION 


25. 


26. 


Worksheet for Pertinent Com- 
ments Concerning the Job ora 
KEYOP 


Continuation Sheet 


* If KEYOP completion fs critical to its 


associated Key Event, place a pound sign 
(#) here. 


4. FMS-RT Key Event Input Form 

An FMS-RT Key Event Input Form is used to enter or to 
change pertinent data concerning shipyard Key Events. This data 
associates specific dates to Key Events and produces “flags” when 
these dates change. This feature aids Shipyard Coordinators in locating 
jobs and KEYOPS that are affected by shipyard schedules and schedule 
changes. Key Event data provided by this form is optional except when 
a Key Event is associated with a critically flagged KEYOP.* 

In this case, Key Event data must have been entered prior to, 


or be entered concurrent with, the “critical” designation. 


* The “critical” flag provides a method to improve interface 
between ship's force work and shipyard work by identifying those 
ship's force work Key Operations (KEYOP) which directly affect 
shipyard Key Events. 
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KEY EVENT INPUT 






EVENT | „ DATE DESCRIPTION 










N [N 

8 
— 

2 


NAS S JS S 
ES IAS N 


NI x 
NS 


LEGEND 3. Key Event Date 
1. Action Code 4. Key Event Description 
2. Key Event 
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5. FMS-RT Material Request Form 

An FMS-RT Material Request Form must be submitted for 
every job requiring material. The form is used to list material required 
for each job and to enter the information into the FMS-RT master file. 
When the related job is approved for accomplishment by ship's force, 
the material may be ordered. The form is also used to change or delete 
data that have already been entered into the FMS-RT master file. 
NOTE; When policy allows FMS-RT Material Requests to be made 
using NAVSUP 1250-1 or DD1348, these forms may be substituted 


(see following pages). 


Te 


FMO MATERIAL REQUEST 4700/8MR TEOT (12/00) 





LEGEND 


1. 


2. 
3. 
4 


om 


Work Center 

Job Sequence Number (JSN) 
Action Code 

Line Item Number (blank unless 
making a change) 
Documentation Number 
Common Name of Material 


Signal Code (used by Supply 
Department) 


COG (first two characters of 
national stock number) 
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10. 


p 
12. 
13. 
14. 


15. 


NSN, or manufacturer's part 
number 


Special Material Identification 
Code 


Unit of Issue 
Quantity 
Unit Price 


Pertinent Comments on the 
Status 


Signatures 


6. NAVSUP Form 1250-1 Material Requisition Document 
A legible copy of a NAVSUP Form 1250-1 may be used to 
identify material required for each job and may be used to enter the 
information into the FMS-RT master file. Elements required to gener- 


ate a FMS-RT material listing are listed in the legend. 
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LEGEND 8. (Block 25) Quantity 

1. (Block 8) Noun Name 9. (Block 26) Unit Price 

2. (Block 14) Work Center 10. (Blocks 36-43) Julian Date 

3. (Block 15) Job Sequence Number NOTESTOEOS A bere Wi 
: Ina on, work centers 

4. (Block 18) Signal Code continue to complete all ele- 

9. (Block 19) COG ments of DD NAVSUP Form 

6. (Blocks 21-22) Stack Number 1250-1 required by the individ- 

7. (Block 24) Unit of Issue ual ship’s Supply Department. 
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7. DD1348 Material Request Form 
A legible copy of a DD Form 1348 may be used to identify 
material required for each job and may be used to enter the informa- 
tion into the FMS-RT master file. Elements required to generate a 


FMS-RT material listing are shown in the legend. 
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LEGEND 8. (Columns 55-56) COG 
1. (Block C) Noun Name 9. (Block T) Unit Price 
2. (Columns 8-20) Stock Number 
3. (Columns 23-24) Unit of Issue TO A E 
4. (Columns 25-29) Quantity ments of DD Form 1348 as 
5. (Column 51) Signal Code required by the individual ship's 
6. (Block M) Work Center Supply Department. 
7. (Block N) Job Sequence Number 
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8. FMS-RT Manpower Planning Form 
This form provides information on the expected level of 
manpower for each work center for every week of the availability. Any 


changes to the original estimates are also made using this form. 
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ACTION CODES 
A - 40D 
C - CHANGE 
2- 2APP.5 


MANPOWER PLANNING 


PREPARED Bv, 1 ——— l il l l ii L i Ë s 5 


WORK a) an ` fl, INDUST % 


7 


Mt PER WORK WEEK LL LLL rrt —M——— RANGE CONSTANT 


s s ACTION CODE Z WILL DELETE WORK CENTER 


11 


LEGEND 7. Range ol Weeks for Men per Week 
1. Name of Originator 8. Range Constant of Men per Week 
2. Date Prepared for designated weeks 
3. Work Center Code 9. Envelope of Time (from week to 
4. Department week) 
l 10. Number of Men Available in 
5. Percentage of Industrial 
those weeks 
Availability SA 
6. Number cf Men per Work Week D PO rss 
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9. FMS-RT Material Status Form 
This form is for the use of the Supply Coordinator only. It is 
used to put current information on material status into the FMS-RT 


master file. 


FM MATERIAL STATUS 4790/8448 TEST (12-86) 


FMS MATERIAL STATUS SCH 


blend e [eres e] ecos [or cloro] corro 





LEGEND 


Document Number 


Pertinent Comments on the 
Status 


7. Quantity Received 
1. Action Code 8. Actual Cost of Materia] Received 
2. Work Center Code 9. Quantity Cancelled 
3. Job Sequence Number 10. Quantity Issued 
4. Line Item Number 11. Storage/Location/Bin 
5. 
6. 
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B. FMS-RT OUTPUT REPORTS 

The FMS-RT, being a real-time system, has been developed with 
the idea that the basic reports are to be generated and excerpts from 
those reports will be available for the Availability Managers on an as- 
needed and demand basis. These reports may be modified or drawn on 
a selectability basis from the Mission Essential work at a work center 
level to that of a total work package for the entire ship. The only limi- 
tations are those of the needs of the Availability Managers. The basic 
reports are as follows. 

1. Reports 


° Session Register. A report that indicates all the entries and 
actions taking place in the last update to the database. 


° Work List w/Material. A report that indicates all of the work, 
Keyops, and material specified for a single work center, a range of 
work centers, or the entire ship. 


° Work List w/o Material. Same report as with material. 


° Material Management Report. This report is a material report 
that can be drawn by document number and/or by NSN stock 
number. This also shows stowage location. 


e Scheduling GANTT Report. This report is used by the ship's 
work center supervisors and/or the personnel selected to accom- 
plish the scheduling of the ship's force work package. 


° Key Event List. This is a listing of the Key Events that will be 
used by the overhauling activity to schedule its work package and 
provided by the activitys Scheduling Department. 


e Workload Forecast and Summary. This is a report that indicates 
to the Division Officer or the Availability Manager the capability of 
a work center for all or part of an availability. This report will 
indicate those capabilities in both man-hours and man-days. 
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° Department Projection. This is a line graph projection of the 
selected department's capabilities (manpower) and workload 
(schedule work) for a 15-week window, selectable. 


e Accomplishing Work Center Report. This is the report that the 
work center will use as an input form to indicate accomplishment 
of jobs and/or Keyops, 4-week window. 


e Department Status. This is a bar graph/pie chart that indicates 
at the department level the capabilities, liabilities, and accom- 
plishments of the selected department. 


e Commanding Officer's Status. This report is a ship-level report 
that indicates at the ship level the liabilities, assets, and accom- 
plishments of the ship's force. 


e Ship's Status. This is a status report only and cannot be updated 
as an individual report. It is meant for a quick output for the 
ship's Commanding Officer and/or the Availability Manager. This 
report is updated continually as the database is changed in any 
way and could be considered the most current data available. 

The above-listed reports are those that are available from the 
system on an as-is basis. If there are unique requirements that the 
managers need for command reports and/or quick looks, the above 
reports may be selectively printed. There is also being considered, for 
the future, a selectable report writer that will allow the individual 
manager to design a report as his individual needs dictate. This will be 
in level II of the FMS-RT and input from the user will be required. If 


there are immediate needs for unique reports, contact your PERA 


Representative or the FMS-RT Program Manager. 
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2. Scheduling GANTT Chart 
a. Report Narrative 
The purpose of this report is to provide a planning tool 
used by the work center supervisor to initially schedule work over the 
length of the availability. In addition, the report provides information 
for the following: 
e Ship's overhaul scheduled start and completion dates 
e Key operation man-hours estimated 
e Key operation scheduled start and completion weeks (if available) 
e Key operation key event (if available) 
e Job key event 
e Job priority 
b. Report Design 
The report is produced in accomplishing work center 
sequence. The report displays all weeks covering the length of the 
overhaul. 
Further report specifications are as follows: 


e Headers and trailers are standard (except report title and selec- 
tion data) 


e If schedule start and completion weeks are available for a key 
operation, they will be displayed on the report; otherwise, dashes 
will appear. 

e Computed items are as follows: 


- Man-hours remaining 


— Man-hours used 
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c. Report Selection 


The user may select a single work center, a string of 


work centers, a range of work centers, or any combination of work 


centers. With the exception of deferred jobs and finished key-ops or 


jobs, the report will display all other jobs and key-ops. 


d. Field Descriptions 


With the exceptions noted below, descriptions of the 


report headers and trailers are in the specifications for standard 


report headers and trailers. 


Item Column Input 
Number Heading Mnemonic 
1 STD-HDR1-WC WC-WC 
STD-TRL2- 
REPORT LEVEL 
2 SELEC 
3 SEQUENCE 
4 STD-HD3-TITLE 
STD-TRL2-TITLE 
5 JCN JOB-JCN 
6 Noun name JOB-NOUN-NAME 
T Iden No. JOB-IES-NR 
8 Kop KOP-KOP-NR 
9 Description KOP-DESC 
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Description 
of Data 


Work center designators 

Work center code is from the 
work center record; also place 
the value “WORK CENTER” here 
Report selection 

All selections (no deferred or 
finished) 

AWC only 


Place the value “SCHEDULING 
GANTT CHART” here 


Job control number 
Description of the job 


Identification/Equipment 
Serial Number 


Key operation number ranges 
from 001 to 999; makes the kops 
unique 


Describes the key operation 


Item 
Number 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


Column 
Heading 


Week SS SC 


M/H Est 


PRI- 


Kop Kev 


QA 


AWC 


Compartment 


Input 
Mnemonic 


KOP-SW 
KOP-CW 


KOP-MHEST 


JOB-PRI 


JOB-KEV 


KOP-KEV 


KOP-QA 


KOP-AWC 


KOP-CMPT-NR 


S 


Description 
of Data 


Week SS is the scheduled start 
week of the kop; week SC is the 
scheduled completion week of 
the kop: if one is present, you 
must have the other. 


Man-hours estimated to com- 
plete the kop; value determined 
by the following formula: MAN- 
HOURS ESTIMATED = MAN- 
HOURS USED TO DATE + MAN- 
HOURS REMAINING 


Job priority ranges from 1 to 4; 
job priority is entered by the 
work center Supervisor 


Contains the key event code; all 
key event codes entering the sys- 
tem must match the key event 
table; maximum 999 key events. 


Key event entered by PERA or the 
SHIP; all the key events must 
match the key event table 


The data is entered by the Ship 
or PERA 


Month and Sunday date of a 
week in a 16-week period in the 
availability 


Number of each week relative to 
the beginning of the availability 


Have to print dashes; they repre- 
sent the separation of jobs 


Accomplishing work center 


Field is broken down by deck 
frame, use; an alpha character 
in leftmost position, then the 
field will not be considered as a 
unit and not separated into the 
deck, frame, use field; if other 
than alpha, the fields will be 
broken down and separated by (-) 
when they are printed 


OWE, START 1/ NW ñ 
OWL COML 16 WN 95 

JS, SS 2. 
WER 


ATTN: WOT STARTED 
290230P 63113-020 
2302 30P 63411-211 


ESNTL 


CANCEL QTY FEDRDERED 52610£931Q 
1  EXESS RICEIVED 
AL STRTUB 
BULK ORDERED 52100294 
BULK ORDERED 5219F395 
RM DATE/TIME 19 OST $5 01:00 
FILE DATE/TIME 18 DEC $5 01:10 


.- El 


USS HARRY U. HILL (DO 906) 


GER j ER ~ E? 
E us Ne i i 
|, GBR Bast i mr LN 
E GER Pasu E E se q Sp 
š = Ë 
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2. Work List/Material 
a. Material Information 

e Line Item Number 

e Document Number (TYCOM fund code- 5th position) 

e National Stock Number (first two positions are the COG code) 

e Nomenclature 

e Quantity Ordered 

e Unit of Issue 

e Quantity Received 

e Quantity Issued 

e Quantity Canceled 

e Material Status 

b. Report Design 
The report is produced in originating work center 

sequence. The report can be produced for the following data 
categories: 

e All jobs regardless of status 

° All active jobs 

° All inactive jobs and/or keyops 

° Mission essential jobs 

° Deferred jobs only 
Headers and trailers are standard (except report title and selection 


data. When a job line is printed, it will be followed by its kop informa- 
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tion and associated material information. Computed items are the 


following: 
e Job physical progress 
e KOP current man-hours remaining 


c. Report Selection 


The report can be produced for a single OWC, a range of 


OWCs, or a string of OWCs. 
d. Field Descriptions 


Descriptions are as follows: 


Item Column Input 
Number Heading Mnemonic 
1 STD-HDR1-WC WC-WC 
STD-TRL2- 
REPORT LEVEL 
2 SEQUENCE 
3 STD-HD3-TITLE 


STD-TRL2-TITLE 


+ PROG/WGHT JOB-PHY-PROG 


KOP-PHY-PROG 


5 JCN JOB-JCN 
6 NOUN NAME JOB-NOUN-NAME 
7 IDENT NO. JOB-IES-NR 
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Description 
of Report Elements 


Work center designators 

Work center code is from the 
work center record; also place 
the value “WORK CENTER” here 


Sequence is OWC 


Place the value “WORK/LIST 
MATERIAL” here 


Physical Progress of a job is the 
sum of the weight of its finished 
kops 


KOP Physical Progress is the 
weight of the kops; when kop is 
finished its weight is added to 
the job line and the work 
“DONE” is printed in its place on 
the kop line 


Job control number 
Description of the job 


Identification/Equipment 
Serial Number 


Item 
Number 


10 


11 


12 


13 


14 


15 


16 


17 


Week SS SC 


Column 
Heading 


M/H EST 


TOT USED 


M/H REMN 


QA 


SCREENING 


MISSN ESNTL 


INSURV NUMBER 


ESWBS/JON 


AWC 


KOP-SW 
KOP-CW 


Input 
Mnemonic 


KOP-MHEST 


KOP-CUM-MHUSED 


KOP-CUR-MHREMN 
(calculation) 


KOP-QA 


JOB-SCREEN 


JOB-ME-CD 


JOB-INSURV 


JOB-ESWBS-JON 


KOP-AWC 


St 


Week SS is the scheduled start 
week of the kop; week SC is the 
scheduled completion week of 
the kop 


Description 
of Report Elements 


Man-hours estimated to com- 
plete the kop; value determined 
by the following formula: MAN- 
HOURS ESTIMATED = MAN- 
HOURS USED TO DATE + MAN- 
HOURS REMAINING 


Sum of all man-hours used on 
the kop 


Man-hours remaining is calcu- 
lated by subtracting KOP-CUM- 
MHUSED from KOP-CUR- 
MHEST, which gives the KOP- 
CUR-MHREMN field 


The data is entered by PERA or 
by the Ship 


This is a comment entered by 
PERA or by the Ship 


Mission essential code identifies 
a job as mission essential; it 
consists of the department code 
and a three-digit number rang- 
ing from 100 to 199; mission 
essential jobs must stay in 
active status until the mission 
essential code is changed 


Inspection and survey INSURV 
(used by SARP) 


Expanded ship work breakdown 
structure formerly known as 
SWLIN (SARP) JON private ship- 
yard job order number used by 
private shipyards 


Work center that will do the 
work on the kop; all AWCs 
entering the system will have to 
match the work center table; 


18 


Item 
Number 


19 


20 


21 


22 


23 


24 


29 


KOP 


Column 
Heading 


DESCRIPTION 


COMPARTMENT 


PRI- 


RMKS 


LIN 


DOC NO. 


NSN 


KOP-KOP-NR 


Input 
Mnemonic 


KOP-DESC 


KOP-CMPT-NR 


JOB-PRI 


JOB-REMARKS 


MTRL-LN-NR 


MTRL-DOC-NR 


MTRL-NSN 
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kops with man-hours expended 
against it will not be able to 
change their AWC 


Key operation number ranges 
from 001 to 999; makes the kops 
unique 


Description 
of Report Elements 


Description of the key operation 


Field is broken down by deck 
frame, use; an alpha character 
in leftmost position means this 
is a comment field and will not 
be separated; any other type 
character in leftmost position; 
the fields will be broken down 
and separated by (-) when they 
are printed 


Job priority ranges from 1 to 4; 
job priority is entered by the 
work center supervisor 


Comments pertaining to the job 
entered by PERA or the Ship 


The line Number contains a 
number generated by the com- 
puter that is used to identify 
material records when no docu- 
ment number has been assigned; 
this number comes from the 
Next Available Line Item Num- 
ber field in the job record; line 
numbers cannot be changed or 
reused 


The Document Number is a 
unique number that is assigned 
by the supply officer at the time 
material is ordered; the fifth 
position is the TYCOM Fund 
Code 


Uniquely identifies the material 
in the supply system (national 
stock number); positions 1 and 2 
of this field make up the COG 
code 
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Item 
Number 


27 


28 


29 


30 


31 


32 


NOMENCLATURE 


Column 
Heading 


QTY ORD 


UI 


QTY RECD 


QTY ISS 


QTY CAN 


MATERIAL 
STATUS 


ATTN: 


MTRL-NOMEN 


Input 
Mnemonic 


MTRL-QTY-REQD 


MTRL-UI 


MTRL-QTY-RCVD 


MTRL-QTY-ISS 


MTRL-QTY-CAN 


MTRL-STAT 


JOB-KEV 


KOP-ATTN 


93 


Description of the material 
itself (e.g., nut, bolt, paintbrush) 


Description 
of Report Elements 


Quantity of material required to 
do the job; used with the unit 
price field to produce the 
extended estimated cost which 
becomes obligated cost when 
material is ordered 


The Unit of Issue code describes 
the lowest unit that can be issued 
or purchased (e.g., EA=each; 
BS=box; CN=carton, etc.) 


Contains a count of the material 
items received to date 


Contains the number of mate- 
rial items issued to the work 
center 


Contains the number of mate- 
rial items cancelled 


Contains either a status code 
which can be interpreted by a 
status look-up table or a 30- 
character comment if code does 
not match look-up table; it will 
be treated as a comment 


Contains the job key event code; 
all key event codes entering the 
system must match the key 
event table; maximum 999 key 
events 


Comment field; three messages 
are system generated: “NOT 
STARTED," “NOT COM- 
PLETED," OR “NOT SCHED- 
ULED." These messages will 
appear when the following con- 
ditions exist: a kop schedule 
start on schedule complete week 
has passed without man-hours 
being expended and the overhaul 
has begun and the kop is NOT 


Item 
Number 


35 


36 


37 


Column 
Heading 


WEEK USED 


Input 
Mnemonic 


JOB-PSYD-JON 


KOP-KEV 
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YET SCHEDULED, however the 
ship can override their messages 


Description 
of Report Elements 


Private shipyard job order num- 
ber used by shipyard; three- 
character field that follows the 
ESWBS 


Key event; entered by PERA or 
the Ship; all key events must 
match the key event table 


Number of man-hours used the 
previous week 
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4. Material Management Report 
a. Report Narrative 
Material Management Report displays the status of mate- 
rial, quantity ordered, quantity received, quantity issued, and 
accounting data for FMS-RT funds. 
b. Report Design 
The report may be obtained in the following sequences: 
e Job Control Number (JCN) 
e National Stock Number (NSN) 
e Document/Requisition Number (DOC) 


¢ Type Commander Fund Code (TYCOM FUND CODE) (with sec- 
ondary sequence on document/requisition number) 


The report may be produced for the following data categories: 

e All material 

e All requisitioned material 

e All outstanding material 

e All material not requisitioned 
Headers and trailers are standard. Material items without document 
numbers sort at the end of the report when requested in that 
sequence. 

c. Report Selection 
The report can be produced for a single work center or 


for all work centers. 
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Item 
Number 


10 


11 


12 


13 


14 


15 


Column 
Heading 


SELECT 


SEQUENCE 


TITLE 


JCN 


LIN 


DOC NR DAT SER 


NSN 
Nomenclature 


QTY ORD 
UNIT OF ISSUE 


QTY RECD 
QTY ISSD 


MTRL STATUS 


ESTIMATED COST 


OBLIGATED COST 


d. Field Descriptions 


Input 
Mnemonic 


STD-HDR2- 
SELECT 


STD-HDR3- 
SEQUENCE 


STD-HDR2-TITLE 


MTRL-JCN 


MTRL-LN-NR 


MTRL-DOC-NR 


MTRL-NSN 
MTRL-NOMEN 
MTRL-QTY-REQ 


MTRL-UI 


MTRL-QTY-RCVD 
MTRL-QTY-ISS 
MTRL-STAT 


MTRL-EXTD- 
EST-COST 


MTRL-OBL-COST 
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Description 
of Data 


A description of the data selected 
to appear on the report 


A description of the report 
sequence 


Move “material management 
report” to this field 


Job control number 


Line item number assigned by 
computer 


Material document number; the 
number under which the item 
was ordered 


National stock number 
Name of part 
Quantity ordered 


Units in which material is 
dispensed 


Quantity received 
Quantity issued 
Status of material 


Net estimated cost of each line 
item 


To have any data shown in this 
field, a document number must 
be present; if no material has 
been received, then obligated 
cost equals estimated cost; upon 
receipt of material, obligated 
cost is decreased by the entry of 
actual cost data; obligated cost 
will equal zero upon receipt of 
all ordered material 


Item 
Number 


16 


17 


18 


19 


Column 
Heading 


ACTUAL COST 


TOTAL EST COST 


TOTAL OBL COST 


TOTAL ACT COST 


Mnemonic 


MTRL-ACTL-COST 


N/A 


N/A 


N/A 


Input 
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Description 
of Data 


Net cost as returned in billing 
Total estimated cost of all mate- 
rial items for the specific 


sequence ordered 


Total obligated cost of all mate- 
rial items 


Total actual cost of all material 
items 
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5. Report—Material Drawdown Request Form 
a. Report Narrative 
The Material Drawdown Request Form report is used to 
draw supplies from the ship's supply storeroom. Material for each job 
will be printed on a separate page. 
b. Report Design 
The report can only be obtained by Job Control Number. 
Only material with a document/requisition number will appear on this 
report. The material items on the report are in document/requisition 
number sequence. Report headers are unique to this report. Report 
trailers are standard. 
c. Report Selection 
The report may be produced for a single job number, a 
string of job numbers, a group of job numbers, or all job numbers. 


d. Field Descriptions 


Item Column Input Identification 
Number Heading Mnemonic of Data 
1 NON-STD-HD3- Move the title of the report to 
TITLE this item; in this case, it would 


be “FMS-RT Material Drawdown 
Request Form," which is not a 
standard header 


STD-TRL1-TITLE Move the title of the report to 
this item; in this case, it would 
be “FMS-RT Material Drawdown 
Request Form” 


2 JOB CONTROLNR . JOB-JCN The Job Control Number is 
assigned by the Originating 
Work Center; it uniquely identi- 
fies jobs; it is composed of the 
Originating Work Center and 
Job Sequence Number 
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Item 
Number 


10 


11 


Column 
Heading 


LINE NR 


DOCUMENT NR 


NSN 


NOMENCLATURE 


U/I 


QUANTITY 
ORDERED 


QUANTITY 
CANCELLED 
QUANTITY 
RECEIVED 


QUANTITY 
DESIRED 


Input 
Mnemonic 


MTR-LN-NR 


MTRL-DOC-NR 


MTRL-NSN 


MTRL-NOMEN 


MTRL-UI 


MTRL-QTY-ORD 


MTRL-QTY-CAN 


MTRL-QTY-RCVD 


N/A 
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Identification 
of Data 


The Line Number contains a 
number generated by the com- 
puter that is used to identify 
material records when no docu- 
ment number has been assigned; 
this number comes from the 
Next Available Line Item Num- 
ber field in the job record; line 
numbers cannot be changed or 
reused. 


The Document Number is a 
unique number that is assigned 
by the supply officer at the time 
material is ordered; the fifth 
position is the TYCOM Fund 
Code 


This is the National Stock Num- 
ber, which uniquely identifies 
the material in the supply sys- 
tem; the first two characters of 
the NSN are the (COG Code); the 
Cognizance Code identifies the 
inventory manager for the 
material 


Description of the material 
itself (e.g., nut, bolt, paintbrush) 


The Unit of Issue code describes 
the lowest unit that can be issued 
or purchased (e.g., EA-each; 
BX-box; CN-carton, etc.) 


This number is used to show the 
quantity of material required to 
do the job 


Number of material items that 
were cancelled 


This is the number of material 
items received to date by the 
ship's supply officer 


Quantity Desired is the amount 
of material items needed by the 
work center supervisor at this 
time 


Item 
Number 


12 


13 


Column Input 
Heading Mnemonic 


QUANTITY ISSUED MTRL-QTY-ISS 


WORK CENTER CONTACT 
LOC/BIN 

SFWP Material Coordinator 
SFWP ADP Coordinator 
SFWP ADP Coordinator 
Warehouse Coordinator 
Work Center Receipt 
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Identification 
of Data 


Quantity issued is the number of 
materia] items issued to the 
work center supervisor 


The work center's supervisor 
and the supply coordinator's 
signature go on the appropriate 
lines 
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6. Workload Forecast and Summary Report Specifications 
a. Report Narrative 
The purpose of this report is to provide the work center 
supervisor with a detailed short-range or long-range view of his avail- 
able manpower versus scheduled work. In addition, the report pro- 
vides a summary in man-days of the following: 
e Remaining man-days available to do work 
e Remaining work (in man-days) that has been scheduled 
e Remaining work (in man-days) that has not been scheduled 
e Work (in man-days) that has been deferred 
e Work (in man-days) that has been accomplished to date 
b. Report Design 
The report is produced in accomplishing work center 
sequence. Reports can be selected by individual work centers and/or 
departments, or they can be selected by a range of work centers 
and/or departments. In addition, the report can be selected by a range 
of weeks beginning with the current week or beginning with a week in 


the future. Report selections for weeks earlier than the current week 


are invalid. 
Item Column Input Description 
Number Heading Mnemonic of Report Element 
1 STD-HDR1- Work center ignation 
REPORT LEVEL WC-WC Work center code is from the 
STD-TRL2- work center record; also place 
REPORT LEVEL |. WC-WC the value "WORK CENTER" here 
2 STD-HD3-TITLE N/A Place the value "WORK LOAD 
STD-TRL1-TITLE N/A FORECAST AND SUMMARY” 


here 
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Item 
Number 


10 


11 


12 


13 


Column 
Heading 
TOTALS 


JOBS 


KEYOPS 


IND M/D 


INDUSTRIAL % 


MAN-HOURS 
PER WEEK 


AVAIL REMN 
(IND M/D) 
SCHED REMN 


(KEYOPS) 


SCHED REMN 
(IND M/D) 


UNSCHED REMN 
(JOBS) 


UNSCHED REMN 
(KEYOPS) 


Input 
Mnemonic 
N/A 


N/A 


N/A 


N/A 


WC-IND-PCT- 
CNSTNT 


WC-MH-PER-WK 


WC-MHAVAIL 


WC-ACTV-KOP-CNT 


WC-MHSCHED 


WC-INACTV- 
JOB-CNT 


WC-INACTV- 
KOP-CNT 
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Description 
of Report Element 


Category of totals at the work 
center level 


Count of jobs that are unsched- 
uled or deferred 


Count of keyops that are sched- 
uled and unfinished, and count 
of keyops that are unscheduled 


Convert man-hours to man-days 
for each Total Category 


Percentage of TOTAL M/H 
AVAIL per week that will be used 
for IND M/H AVAIL 


The number of man-hours per 
week that each man is scheduled 
to work 


This is a calculation of the num- 
ber of man-days available from 
the current week of overhaul to 
the end of overhaul; formula: 
WC-MHAVAIL/8 - AVAIL REMN 


This is a count of the number of 
scheduled key operations that 
are not finished or deferred 


This is a calculation of the num- 
ber of man-days scheduled; 
deferred and/or finished man- 
days are not included; formula: 
WC-MHSCHED = SCHED REMN 
(IND M/D) 


This is a count of the number of 
jobs that are unscheduled and 
not deferred 


This is a count of the number of 
key operations that are 
unscheduled and not deferred 


Item 
Number 


14 


15 


16 


17 


18 


19 


20 


2l 


Column Input Description 


Heading Mnemonic of Report Element 
UNSCHED REMN WC-MHEST This is a calculation of the num- 
(IND M/D) WC-MHSCHED ber of unscheduled man-days; 


deferred man-days are not 
included; formula: WC-MHEST - 
WC-MHSCHED/8 = UNSCHED 
REMN (IND M/D) 


DEFERRED (JOBS) WC-DFR-JOB-CNT This is a count of the number of 
jobs that have been deferred 


DEFERRED WC-MHDFR This is a count of the number of 
(IND M/D) man-days that have been 
deferred; formula: WC- 
MHDFR/8 = DEFERRED (IND 

M/D) 


USED (IND M/D) WC-MHUSED This is a calculation of the num- 
ber of man-days that have been 
used; formula: WC-MHUSED/8 = 
USED (IND M/D) 


N/A N/A Week of overhaul 


N/A N/A I-LINE; weeks on the left of the I- 
LINE are history; the week to the 
immediate right of the I-LINE is 
the current week; all other weeks 
to the right of the I-LINE repre- 
sent the future 


N/A N/A Title “M/H SCHEDULE” all data 
below this line are in man-hours 


IND M/H SCHED KOP-SW 
KOP-CW 
KOP-CUR-MHREMN 
KOP-WK-N1-MHSCHED 
KOP-WK-N2-MHSCHED 
KOP-WK-N3-MHSCHED 


This is a calculation, by week, of all remaining man-hours scheduled in the 
work center. Deferred jobs are not included. Only key operations that meet 
the selection criteria for the report will be used. The man-hours remaining 
(KOP-CUR-MHREMN) for each key operation are pro-rated within the range 
of its start and completion weeks. The pro-rated man-hours are then added 
to their respective week counters in an array of schedule week counters. 
When the hours for all key operations have been added to the array of 
industrial man-hours scheduled counters, the array is printed on the report 
to the right of the I-LINE (see NOTE for an explanation of key operation 
scheduling and bow-waving). Historical data will not be printed on the 
report unless the current week (where current week is greater than 1) and the 
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Item 
Number 


22 


23 


24 


Item 
Number 


25 


I-LINE is also printed. When historical data is printed, calculations are as 
follows: 


For each selected key operation, man-hours scheduled for the three weeks 
prior to the current week (KOP-WK-N1-MHSCHED, KOP-WK-N2 MHSCHED, 
KOP-WK-N3-MHSCHED) are added to their respective week counters in an 
array of industrial man-hours scheduled counters. The array is then 
printed on the report to the left of the I-LINE. 


Column Input Description 
Heading Mnemonic of Report Element 
IND M/H AVAIL WC-MH-PER-WK With the exception of weeks in 
WC-IND-PCT- the past, this calculation is 
CNSTNT made for each week of the over- 


WC-NR-MEN-WK haul. The calculation is as fol- 
lows: for each week multiply 
TOTAL M/H AVAIL (subscripted) 
by (INDUSTRIAL 96 x .01), giving 
IND M/H AVAIL (subscripted) or 
WC-NR-MEN-WK (subscripted) x 
(WC-IND-PCT-CNST x WC-MH- 


PER-WK x .01) 
DIFFERENCE IND M/H SCHED With the exception of weeks in 
IND M/H AVAIL the past, this calculation is 


made for each week of the over- 
haul; the calculation is as fol- 
lows: subtract IND M/H SCHED 
(subscripted) from IND M/H 
AVAIL subscripted); negative 
totals are acceptable 


IND M/H USED KOP-WK-N1-MHUSED 
KOP-WK-N2-MHUSED 
KOP-WK-N3-MHUSED 
KOP-MHUSED-THIS-WK 


This calculation is made for the current week and the three previous weeks; 
for each selected key operation. Man-hours used for the current week (KOP- 
MHUSED-THIS-WK) and for the three weeks prior to the current week (KOP- 
WK-N1-MHUSED, KOP-WK-N2-MHUSED, KOP-WK-N3-MHUSED) are added 
to their respective counters in an array of Man-Hours Used counters; the 
array is then printed on the report to the left and immediate right of the 
I-LINE. 


Column Input Description 
Heading Mnemonic of Report Element 
NR OF MEN WC-NR-MEN-WK This element is not printed for 


weeks in the past; for each week, 
print the value found in men 
count per week (WC-NR-MEN- 
WK) (subscripted) 
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Item 
Number 


26 


Column 
Heading 


TOTAL M/H 
AVAIL 


Input 
Mnemonic 


WC-NR-MEN-WK 
WC-MH-PER-WK 
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Description 
of Report Element 


This calculation is made for the 
current week, the three previous 
weeks, and for future weeks; for 
each week, multiply the number 
of men per week by the man- 
hours per work week constant or 
WC-NR-MEN-WK (subscripted) x 
WC-MH-PER-WK = Total M/H 
AVAIL (subscripted) 
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Key Event List Report Specification 


a. Report Narrative 


The objective of the Key Event List report is to provide a 


b. Report Design 


managerial tool to keep track of the occurrence of key events. 


The report is produced in key event number sequence or 


report title sequence and report level). 


c. Report Selection 


in key event date sequence. Headers and trailers are standard (except 


All key events are selected for this report 


d. Field Descriptions 


Column Input 
Heading Mnemonic 
STD-HDRI- 
REPORT-LEVEL 
STD-TRL2- 


REPORT-LEVEL 


STD-HDR3-SEQUENCE 


STD-HDR3-TITLE 
STD-TRL1-TITLE 


KEY EVENT NR KEV-KEV-CD 
KEY EVENT KEV-DESC 
DESCRIPTION 
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Identification 
of Data 


Move the report title to this 
item; in this case it is “SHIP” 


Move the report sequence to this 
item; in this case it is “Key Event 
Number” 


Move the report title to these 
items; in this case it is “Key 
Event List” 


The KEY EVENT NR contains a 
four-position code obtained 
from the shipyard or a code 
arbitrarily assigned by the ship- 
yard to represent important 
milestones in the overhaul 
period 


This field contains a brief 
description of the key event 


Item 
Number 


10 


11 


D 


Column 
Heading 


KEY EVENT WEEK 


CURRENT KEY 
EVENT DATE 


WEEK KEY EVENT 
LAST CHANGED 


PREVIOUS KEY 
EVENT DATE 


SY/SOS 


SY SCHEDULER 


AVAIL MANAGER 


Input 
Mnemonic 


KEV-WK 


KEV-DTE-MIL 


KEV-PRV-DTE-MIL 


Identification 
of Data 


The KEY EVENT WEEK contains 
the week of overhaul in which 
the key event will occur; it is 
determined by using the over- 
haul start date and the key event 
date fields; obtain the difference 
between the dates and convert it 
to a week number 


This is the date that the key 
event is scheduled to occur 


The WEEK KEY EVENT LAST 
CHANGED field is the week the 
key event was last changed; to 
find this week, convert the 
modification date in the key 
event record to weeks 


This field contains the date of 
the previous key event; if the key 
event date is changed, then the 
superseded date will be moved to 
this field prior to the change 


Reserved for the Supervisor of 
Shipbuilding's signature 


Reserved for the Shipyard 
Scheduler's signature 


Reserved for the Availability 
Manager's signature 
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8. Accomplishing Work Center Report 
a. Report Narrative 
The Accomplishing Work Center Report is the primary 
working document for the work center. During the availability, space 
is provided to report the number of man-hours used, completion of 
keyops, and changes to the scheduled start and completion weeks and 
man-hour estimates. Space is also provided to report man-hours used 
for keyops scheduled in the future. 
b. Report Design 
The report is produced in accomplishing work center 
sequence. It is selected in four-week increments beginning no earlier 
than the current week. 
Further report specifications are as follows: 
e Headers and trailers are standard 
e Reports will be produced for the following data categories: 
- All jobs and keyops in progress 


- All jobs and keyops scheduled to start before and/or during the 
period covered by the report 


- AM jobs with key operations extending beyond the overhaul 
completion date 


c. Report Selection 
The user may select a single accomplishing work center 
or a string of work centers, a range of work centers, or any combina- 
tion of range and string. The report will display only jobs and keyops 


that are scheduled and not deferred or finished. 


Item 
Number 


1 


10 


11 


d. Field Descriptions 


Column 
Heading 


STD-HDR1-WC 
STD-TRL1-TITLE 


STD-HDR3-TITLE 
STD-TRL2- 
REPORT-LEVEL 


PROG/WGHT 


JCN 
NOUN NAME 


INDENT NO. 


WEEK SS SC 


M/H EST 


TOT USED 


M/H REMN 


QA 


Input 
Mnemonic 


WC-WC 
WC-WC 


N/A 


KOP-PHY-PROG 


JOB-PHY-PROG 


JOB-JCN 
JOB-NOUN-NAME 


JOB-IES-NR 


KOP-SW 
KOP-SC 


KOP-CURR-MHEST 


KOP-MHUSED 
TO DATE 


KOP-CUR-MH- 
REMN 


KOP-QA 


Description 
of Data 


Work center designation 
Also place the value "WORK 
CENTER" here 


Place the value “ACCOMPLISH- 
ING WORK CENTER REPORT” 
here 


Keyop level: Assigned by the 
work center supervisor; an esti- 


mate of the relevance of the 
keyop 


Job level: Increments as keyops 
are completed; will never exceed 
100% 


Job control number 


Job title 

Model or serial number MK and 
MOD, that identifies the 
equipment 


Scheduled start (SS) and sched- 
uled completion (SC) weeks for 
keyop 


Estimated man-hours to com- 
plete keyop 


Man-hours used on keyop to date 


Man-hours remaining to com- 
plete keyop 


Quality Assurance "S' to be per- 
formed by ship's force; "Y' to be 
performed by industrial activ- 
ity; “T” to be performed by ten- 
der; blank no QA required 


Item 
Number 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


22 


23 


Column 
Heading 


MONTH 


M/H USED 


SS WK 


SC WK 


M/H REMN 


KOP 


DESCRIPTION 


COMPARTMENT 


WEEK 


WEEK 


Input 
Mnemonic 


N/A 


KOP-MHUSED- 
THIS-WK 


KOP-FNSH-FLG 


N/A 


N/A 


N/A 


KOP-KOP-NR 


KOP-DESC 


KOP-CMPT-NR 


JOB-KEV 


N/A 


N/A 
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Description 
of Data 


Month and Sunday date of a 
four-week period in the avail- 
ability; report is restricted to a 
four-week envelope; ship selects 
the four weeks by designating 
first week 


Man-hours worked on keyop in 
current week 


Finished "F" is inserted in this 
column when keyop has been 
completed 


Scheduled start (SS) weeks for 
keyops; entries are made only if 
changes are required 


Scheduled completion (SC) 
weeks for keyops; entries are 
made only if changes are 
required 


Man-hours remaining to com- 
plete keyop; entries are made 
only if changes are required 


A step in the completion of a job 


Description of work performed 
on keyop 


Space that will be affected when 
the work is being done, used as 
an identifier for S4 interface 


Key event; the key event number 
represents a milestone in the 
availability; key events and 
their dates are normally pro- 
vided by the shipyard/ 
contractor 


Number of each week relative to 
the beginning of the availability 


Number of man-hours scheduled 
for keyop in week under which it 
is listed (man-hours REMN 
divided by scheduled weeks 
remaining to complete keyop 


Item 
Number 


24 


25 


26 


27 


Column 
Heading 


N/A 


PRI 


WRITE-IN 
SECTION 


WEEK USED 


Mnemonic 


N/A 


Input 


JOB-PRI 


N/A 


116 


Description 
of Data 


Question marks (???) indicate a 
jeopardy condition for that 
keyop, e.g., "NOT STARTED," 
"NOT COMPLETED," or "NOT 
SCHEDULED" 


3M system priority or special 
priority established by the 
TYCOM 


When a keyop not scheduled 
during the four weeks covered by 
this report is started; enter JCN 
and keyop number and report 
man-hours used as previously 
described; changes in a man- 
hour est and keyop scheduling 
may also be made if required 


Number of man-hours used the 
previous week 


APPENDIX C 


COVER LETTERS AND SURVEY FORM 


MEMORANDUM 
30 July 1988 


From: LT Michael MANSFIELD, Computer Technologies Office (Code 
37), Naval Postgraduate School, Monterey, CA 93943 


To: Commanding Officer, USS RICHMOND K. TURNER (CG-20) 


Subj: Data Collection on the Usability of the Fleet Management 
System - Real Time (FMS-RT) 


Encl: (1) Ten Copies of a Usability Survey of the FMS-RT System 


1. I respectfully request your commands participation in a 
survey of the usability of the FMS-RT system. Only 25 ships have 
used this system since it entered the fleet in January 1988. 

Your ship is one of only a very few capable of assisting us in 


collecting valid user evidence to help improve this system. 


2. This survey is part of my thesis on the FMS-RT system. PERA 
(CRUDES) Philadelphia is the sponsor of this research. The 
results of this survey will be available through the Postgraduate 
School or through PERA (CRUDES). 


3. I respectfully request that the ten surveys sent as enclosure 
(1) be completed by various members of your command. We desire 
that they be distributed evenly between those who fill out the 
input forms like workcenter supervisors and those who interpret 
and make decisions based on the output like division officers. 

It is important that the people in your command that actually put 
the changes into the computer are part of the ten men surveyed. 


4. We realize your time and your crews time is very valuable and 
is overly burdened during a major overhaul period like the one 
you are presently going through or recently completed, however 
your help is needed to improve this new system. We would 
appreciate your assistance in this data collection effort. 


Very Respectfully, 
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Information for Survey Participants 

The purpose of this survey is to ascertain the strengths and 
weaknesses of the FMS-RT system you are currently using aboard 
your ship. Your name is not important, all that is important is 
that you are responsible for either entering data into the system 
through the various input forms or you are responsible for 
determining what to do based on the output of the system. 

The FMS-RT system entered the fleet in January 1988, 
therefore, it is important to get feedback from the user's of 
strengths, weaknesses, and possible enhancements. Your responses 
will be correlated with other ships within your fleet. We 
anticipate responses form about eight to ten ships per fleet 
(PACFLT and LANTFLT). The results of this thesis will be 
available by request through the Naval Postgraduate School and 
through Naval Sea Systems Command Detachment PERA (CRUDES) 
Philadelphia, PA 19112. Additionally, each surface type 
commander will receive a copy of this report. 

Only about 25 ships have used this system since it reached 
the fleet. Your honest opinion is needed to create valid 
evidence to improve the FMS-RT system for the future. Feel free 
to attach handwritten listings of problems or improvements if you 
run out of room on the last question. What you say does matter 
and can have significant impact on the future of this system. 

The more detailed the information you provide, the better the 
results. 

In summary, the results of this survey will go to the people 
responsible for maintaining the FMS-RT system and your type 
commander. The results will be fleet specific. Your 
participation is essential to make this system better. We know 
your time is valuable, but a little time spent now may Save many 
hours of agony spent by others trying to figure out problems with 
a faulty system. This is your chance to make at least this 


System work better for you. Thank you very much for your cooperation. 
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Usability Survey of the 
Fleet Management System - Real Time (FMS-RT) 


COMMAND : 


The purpose of this study is to measure how you feel about 
certain aspects of the computer-based FMS-RT system you have or 
are currently using. 

On the following pages you will find different factors, each 
relating to some aspect of the FMS-RT system. Please rate each 
factor on the descriptive scales that follow it, based on your 
evaluation of the factor. 


1. What is your position at your command? 
Division Officer 
Chief Petty Officer (E7 - E9) 
Work Center Supervisor 
FMS-RT Data Processor 
Other (specify) 


2. How many major overhauls (ROH’s, PMA’s or SRA’s) have you 
gone through? 


The scale positions are SS as follows: 


adjective X : : : : : : : : adjective Y 
(1) eg (3) (4) (5) ve (7) 

(1) extremely X (os TIGRE Ey Y 

(2) quite X (6) quite Y 

S) slightly X (7) extremely Y 


(4) equally X or Y 


The following example illustrates the scale positions and 
their meanings: 


My vacation in GE was: 
restful : ; : : : : : f heccrc 


healthy : SX š : : : : : : unhealthy 


According to the responses, the person's vacation was 
extremely hectic and quite healthy. 


INSTRUCTIONS 
a. Check each scale in the position that describes your 
evaluation of the factor being judged. 
b. Check every scale; do not omit any question. 
c. Check only one position for each scale. 


THIS, NOT THIS 
d. Check in the space, not between spaces. NA i X : 
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e. Work rapidly. Rely on your first impressions. 
ANSWER BASED ON YOUR FEELINGS. 


ce Degree of training provided on the FMS-RT and overhaul 
process: 

complete incomplete 

Superior : : : : : : : : inferior 


4. Users’ understanding of the objectives of the FMS-RT: 


insufficiente : : : : : : >: sufficient 
complete incomplete 
Se Users” feeling of participation in the overhaul management 


process by using this system: 


positive : : : : t : : : negative 


insufficiente ° : : : : : : sufficient 


6. Attitude toward using the FMS-RT: 


cooperative CIA A Y [[ belligerent 
negative ; — 1 5 et D EOS 
7. Reliability of output information: 
high: — : "SU EN m E M 
complete :  : —:  : AO A Sn Bore 
8. Relevancy of output information to overhaul planning: 
relevant : ? t. 8 O S rr leva e 
useful ; —:. —: "MC NM cose 
9. Accuracy of output information: 
inaccurate : č :  : "IMD UU Jg rare 
certain : : d : : : : : uncertain 


10. Number of people involved in communicating changes in data: 
(Check one) 
Too Many 
Just Right 
Too Few 
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MNT. Number of changes to data per week. Changes include 
addition to, deletion of, and modification of the data. 


12. As a user of the FMS-RT would you like it to be interactive? 
Interactive computer systems are like SNAP I and II where you 
follow the commands on the screen to enter or change data. These 
type systems would allow changes to the ships database just like 
a CSMP addition, deletion or change is made with various level of 
authority for approval. 

YES NO 


13. Do you see a need to see what other ships in your class did 
during their overhauls or do you see each overhaul as a separate 
event with little to no relation to other ship class? 


I see the need to look at other ships data. 
I see no need to look at other ships data. 


14. Are the reports generated in a useful format? 
YES NO 


15. Did your training for overhaul on the FMS-RT teach you how 
to use the output reports as a tool to manage the overhaul 
process? 

YES NO 


16. Do you use another system in addition to FMS-RT to track job 
status like a manual system devised internally? 
YES NO 


17. How do you evaluate the overall usability of the FMS-RT? 
Extremely useful 
Quite useful 
Better than nothing 
More of a problem than a useful tool 


18. What coast did you receive your training on: (Circle one) 
East or West 


19. List any improvements you would like to see made in this 
system or problems encountered when trying to utilize this 
system: 


Thank you very much for your cooperation. 


121 


APPENDIX D 
FMS-RT PROBLEMS AND ENHANCEMENTS 


Question 19 of the usability survey asked the respondent to list 
any improvements he would like to see made to the FMS-RT system or 
any problems he encountered when trying to utilize this system. Of the 
120 survey responses, 76 respondents provided the following list of 
problems and possible enhancements. The remaining respondents did 
not provide an answer to this question. The number in parentheses 
after any of the items indicates the number of respondents who made 
the same or very similar comments; the letter “P” indicates a problem 
in the system and the letter “E” indicates an enhancement to the 


system. 


° Training on FMS-RT was incomplete. (18P) (The researcher 
looked in detail at this problem. Its cause was the transition from 
SFOMS to FMS-RT. The ships already in overhaul had to switch 
systems partway through and did not receive the FMS-RT Phase I 
and II training. They received a one- to two-day, scaled-down 
training program specifically on FMS-RT.) 


e Make the system interactive so that work center supervisors can 
make changes to their part of the database when they have the 
time. During overhaul, work center supervisors must submit their 
reports by a certain deadline and receive reports at a certain time. 
It would be helpful if this system was part of the SNAP system and 
could be used all the time for weekly work schedules, etc. (18E) 


e Incorporate training on how to use the output forms in managing 
the overhaul. More emphasis on how to efficiently use FMS-RT as 
a tool rather than how to read and fill out reports. We need to 
know what the reports are telling us the problems are. (17P) 


° FMS-RT needs a percentage completion field for each job and a 
remarks space to enter detailed information. The system is weak 


122 


at tracking jobs at the end of the overhaul, when more informa- 
tion is needed to provide the commanding officer adequate detail 
to support major decisions, like crew move-aboard and other 
upcoming events. (16E) 


The output reports need to be in a more useful format. The 
reports should be created by a group of users, like commanding 
officers make a generic report form for their use and division offi- 
cers and work center supervisor make up the forms they need to 
use. Get the users to make the standard reports in a more useful 
format. (15P) 


Provide a tutorial so people interested can learn how to use the 
system and its software. The current documentation is not written 
at the level of the normal user. (14E) 


Totally automate the material tracking function. Link the com- 
puter aboard the ship to the Naval Supply Center (NSC) or Ships 
Parts Control Center (SPCC). (12E) (The researcher's limited 
investigation of this enhancement found that the software would 
have to link into the Supply Systems Uniform Data Automated 
Processing System (UDAPS) either at the local NSC or at SPCC. 
The computer would have to use the requisition number as the 
primary field to interface.) 


The form feed on the graphics does not operate correctly. When 
all departments are selected, extra spacing is inserted. (12P) 


The Worklist Form and Worklist/Material Form do not print com- 
partment numbers correctly. They leave out the *-.” (11P) 


The Scheduling Gantt Chart dates across the top do not print 
properly. The calendar may print six weeks of December when 
there were only five and the sixth week should have been January. 
The number of the day is correct, but the month function has a 
tracking problem. (11P) 


In SFOMS, they created an initial SFWP and the crew corrected 
that and added or deleted jobs as applicable. This was helpful in 
getting the ship started because they could look at their jobs. 
They had reports that applied to them. This made acceptance of a 
new system easier than the FMS-RT system, where the ship has 
to create the initial SFWP and correct it as time passes. As an 
enhancement, create that initial SFWP. (1OE) 
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The printer dip-switch settings are not provided to the user 
either in the FMS-RT Operating Manual or in the FMS-RT 
Administrative Manual. (10P) 


Error messages do not stay on the screen long enough. (10P) 


The system has no “what if” or query capabilities and has very 
limited sort capabilities. (9E) 


Have FMS-RT provide the overhaul SITREP. The SITREP can be 
made into a standard form by post-overhaul commanding officers 
and the type commanders. (9E) 


Prepare a report of all jobs linked to a particular key event, like 
the Light-Off Examination. This permits concentration of effort in 
meeting critical shipyard events. (8E) 


When control-F10 is selected for producing all department 
charts, the system loops with the same chart being prepared. (8P) 


Add the capability to add a key operation to a finished job, as well 
as the ability to add and finish key operations at one screen. (7P) 


The Workload Forecast and Summary Report does not update 
man-hours scheduled and man-hours used to the left of the I-line. 
(oP) 


Prepare a critical job-tracking form. The form can be limited in 
the number of jobs that can be on it to ensure commands only 
have a limited number of critical jobs. The entry of jobs to the list 
is controlled by the commanding officer or his delegated repre- 
sentative. The list has very detailed information, such as material 
requisition status and current job status in a brief, remarks-type 
format. (4E) 


Ensure all menu selections are consistent. For some menu selec- 
tions you have to hit the enter key, but for others you select the 
number and it automatically takes you to the part of the program 
you want. (3P) 


Do away with the four-week window column on the Workload 
Forecast and Summary Reports. (2E) 


Change the color of the screens. The red screen on a black back- 
ground is disturbing to the eyes. The colors make it hard to do 
changes in one long sitting because you get headaches from look- 
ing at the screens for an extended period of time. (2P) 
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e The dual hard disk is hard to use. Back-ups should be automatic 
when existing from any part of the program. Provide an easy, 
automatic back-up capability. This may need prompts as you exist, 
like “Before exit function is complete, do you want the transac- 
tions from this session saved? Yes or No.” (1P) 


° The documentation on the system is poor. Provide the ship with 
better user manuals and a help line (a phone help service). (1P) 
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10. 


11. 


12. 


13. 


LIST OF REFERENCES 


Webster's Ninth New Collegiate Dictionary, 9th ed., pp. 1251 and 
1299, Merriam-Webster Inc., 1985. 


Webster's 3rd New International Dictionary, 3rd ed., p. 2424, 
Merriam-Webster Inc., 1985. 


Pressman, R. S., Software Engineering: A Practitioner's Approach, 
2nd ed., pp. 452-463, McGraw-Hill, 1987. 


Executive Office of the President, Office of Management and 
Budget, Circular No. A-76 (Revised), Performance of Commercial 
Activities, Government Printing Office, Washington, DC, August 
1983. 


Turban, E., and Meredith, J. R., Fundamentals of Management 
Science, 3rd ed., pp. 319-377, Business Publications Inc., 1985. 


Fleet Management System-Real Time Operator's Guide, not dated. 


Head, R., “Management Information Systems: A Critical 
Appraisal,” Datamation, v. 13, no. 5, pp. 22-28, May 1967. 


Fleet Management System-Real Time Administrative Manual, 
NAVSEA S9801-A-A-SHP-010/FMS-RT, January 1988. 


Baroudi, J. J., and Orlikowski, W. J., “A Short-Form Measure of 
User Information Satisfaction: A Psychometric Evaluation and 
Notes on Use,” Journal of Management Information Systems, v. 4, 
no. 4, pp. 44-59, Sprint 1988. 


Interview with R. Jackson, Envisions Corporation, aboard USS 
Jarett at Naval Station Long Beach, CA, 14 June 1988. 


Interview with CDR Loeffler, USN, Commanding Officer USS 
Hewitt in San Diego, CA, 17 Jun. 


Interview with R. Cordeli, GS-12, SUPSHIPS San Diego, 20 June 
1988. 


Interview with M. Rose, Continental Maritime of San Diego, 21 
June 1988. 


126 


14. 


15. 


16. 


E. 


Fleet Management System-Real Time Work Center Supervisor User's 
Booklet, not dated. 


Stahl, B., "Testing for Usability Can Head Off Disaster," 
Computerworld, pp. 83-92, 7 December 1987. 


Department of Defense Standard DOD-STD-2167A, Defense 
System Software Development, 29 February 1988. 


Department of the Navy Military Standard MIL-STD-1472C Notice 


3, Human Engineering Design Criteria for Military Systems, 
Equipment and Facilities, 17 March 1987. 


J 7 


BIBLIOGRAPHY 


Charet, R. N., Software Engineering Environment: Concepts and 
Techniques, McGrav-Hill, 1986. 


Department of Defense Standard DOD-STD-2168A, Defense Sustems 
Software Quality Program, 29 February 1988. 


Grauer, R. T., and Sugrue, P. K., Microcomputer Applications, McGraw- 
Hill, 1987. 


Harmon, P., “Training: Psychology Meets Technology,” 
Computerworld, v. 17, no. 18, pp. 17-16, 2 May 1983. 


Kerlinger, F. N., Foundations of Behavioral Research, 3rd ed., CBS 
College Publishing, 1986. 


Karadag, O., An Algorithm for Scheduling a Ship’s Force During a 
Shipyard Overhaul, Master’s Thesis, Naval Postgraduate School, 
Monterey, CA, September 1987. 


Liss, S. M., and O'Rourke, S. T., SNAP II: Training Administrative 
Enhancements, Masters Thesis, Naval Postgraduate School, Monterey, 
CA, September 1987. 


Munro, M. C., Huff, S. L., and Moore, G., “Expansion and Control of 
EUC," Journal of Management Information Systems, v. 4, no. 3, pp. 5- 
27, Winter 1987-1988. 


Sanders, M. S., and McCormick, E. J., Human Factors in Engineering 
and Design, 6th ed., McGraw-Hill, 1987. 


Scharer, L. L., "User Training: Less is More," Datamation, pp. 175- 
182, July 1983 


Schott, F., and Olson, M., "Designing Usability in Systems: Driving for 
Normalcy," Datamation, pp. 68-76, 15 May 1988. 


Ship's Force Overhaul Management System Administrative Manual, 
NAVSEA S9081-AA-SHO-010/SFOMS, February 1979, rev. October 
1984. 


Ship's Force Overhaul Management System User's Booklet, rev. 3.5, not 
dated. 


128 


Sprague Jr., R. H., and Watson, H. J., Decision Support Systems: Putting 
Theory into Practice, Prentice-Hall, 1986. 


Thomas, J. C., and Schneider, M. L., Human Factors in Computer 
Systems, vol. I and II, Ablex Publishing Co., 1984. 


129 


INITIAL DISTRIBUTION LIST 


No. Copies 


Defense Technical Information Center 2 
Cameron Station 
Alexandria, VA 22304-6145 


Library, Code 0142 2 
Naval Postgraduate School 
Monterey, CA 93943-5002 


Director, Information Systems (OP-945) 1 
Office of the Chief of Naval Operations 

Navy Department 

Washington, DC 20350-2000 


Superintendent, Naval Postgraduate School 1 
Computer Technology Programs, Code 37 
Monterey, CA 93943-5000 


Commander, Naval Surface Force 1 
U.S. Atlantic Fleet 
Norfolk, VA 23511-6292 


Commander, Naval Surface Force 1 
U.S. Pacific Fleet 

Naval Amphibious Base, Coronado 

San Diego, CA 92155-5035 


Commanding Officer 1 
USS Stein (FF-1065) 
FPO San Francisco, CA 96678-1425 


Director, Naval Sea Systems Command 1 
Automated Data Systems Activity 

Naval Ordnance Station 

Indian Head, MD 20640-5038 


Naval Sea Systems Command Detachment 2 
PERA (CRUDES) 

Philadelphia Naval Shipyard 

Philadelphia, PA 19112 


130 


lO. 


pl. 


12. 


13. 


Surface Warfare Officers School Command 
Newport, RI 02841 


LTC Gerald Pauler, Code 55Pa 
Naval Postgraduate School 
Monterey, CA 93943-5000 


Professor Nancy C. Roberts, Code 54Rc 
Naval Postgraduate School 
Monterey, CA 93943-5000 


LT Michael Mansfield 


3 Seward Street 
Saratoga Springs, NY 12866 


131 











BLO 














Thesis 
M321 
eel 


SS 





Mansfield 

An evaluation of tun 
quality of training and 
usaili of the Fleet 
Management System-Real 
Time (FMS-RT). 





VEMLU 










































hl dome L: ^d 
rr A WEE E AER LKE bh rd aypay. - 
Da PEU hehe RT EST E LE LAL 









PT H T T eia 
AAN e MITT geed A V 66888 Q5 EA AR A 00.0 M 
E NN te A NS O Y 
TTE P iens d PTS ANE ] 
i thesM321 


PA Pub sedie gi Pier aoe ol JM 
Prep Model M RE RAPE EE L DAR AA " 
e n O A PET TA NN 


diala ETA As; 
H N SAMA E k Ç Lar F Du peysa TP NO p 3 e P 
e ` EX An evaluation of the quality of training 


Ptol s 
U AS? PREX IL. 






















` 4 i L 
ee P — AAA AR ARS SE 
A S 895,8 e, T A SA ET b 

AIR YU 


PET eS di Far Y L BEE E 
Per Pe rele PT Jid DC TV SCHIER A EE 
E dd APRA de 414 | i | III! AW | II I | | IH! 
A A E Oe RO A ETC FAT ELS A ; | | | UD LE | 
AAA mining PW eee TC EE A E WE bg yt Ré. (TYNE TTI P E | | | | ll | 
pay EO ret MONA a E uU SRM e * ? x 4 | | | Wh | 
"1 $ | | TU | | | | | | MI | l 
| | | | | | |! I 
| | | MI | | 
' | | l l ui Ui 













A EI edi 
AI Leet éch [I Pur ru ETUR obe yir posu. € nv ^ d.a t 
PARA s s a af 2:3 Seda 0 RA 1Qgbo 6 oos doe Bec AM TT T "o US 
A A enge PRATS AS 4.23401) 4 € Fr E | 
ipso OE de rp LAA TED Sab: STAR E 
REA aan AAA ide HARTA K € 

d al a AS dd AAA A s he 

PPP TA ehh eet "TT, 
yer LS F r 





l 






i 


3 2768 000 84262 9 




















— gegen gg Ler t C —B A 4 
AA el BE PARAR E SD E GE A SATA PU bid D 
nd TE Ee EA) f -aet LE: L a [ 
e DUDLEY KNOX LIBRARY 


A hg [i 





PA A sies 
(CW E TD bo b; 





ppp adi 
API AN au L PUERTA e 
A A A d N PO E k... 4.5355 Wf od af t o . 
A kg A were A a AAA EI eile bere ECH AA "II 
"I AA d C MT PR 
^ 


AA AS PD ded 
cur qon daptgnpsbe deg PO CCEPIT E EU D A L £ 
capped D ipae pe p — par eremum LE des 
A 7 P HE TEE did Io TTT A A L * 
ajeno EE Eh iioi it: pot wer qe yer renda pvp eive ih. T 

PRA at E A ir al vr t up qar qp ayu y apy YY T " 
AAA uot A T TS PAET 
















Ao 
—— — — m PASA TE AO 
P TE e son tal» PASA AT ER mó SE EK RN RER CA OPETE SE) — 
DE ET — E AN RA ds PTE A a Al Pd ee ee — — 1.18 
P TES o e O A aura, CEA BJ nm 
owe TTE A IS ° wr IN Ls py 
ET D 








P dedo E A E E 
o AA k A adits 
AA A RA A A 
NS erepta a A PP EM X E E a aa 
AAA ri PRA TTT SS re M) 
AAA A as dd AAA A EA A AA SAA E 
Fara" api Ae Vie A SS ADA a "M 
Vin mro nd A PRATS TO 
rarer E E Leg GET CH LIE ee 
T T E Y ass spg sua. TTT s , 
BT DH a APRA EK PAPA aA PAS A 
H WA RK 


Boa E ben, Pa Ta y géng DALL 
D EE liang DEEM WO E ED LCD KA, 
CLD # d. aapp paf A 0 0.9 L 


A eh j 
PA ATTE 
AA " 







































er AS ad ° 
d A A AA ns do TIT P 
EAA ape p d E TE ad KD D ia UT 
A A LUE ih aiiis A nA wh, PV TE n METRE. 
Edge REIH E AA 

A A A rodeo] PAS , 
» pne d yv Sr a ri rer Par ee Y E 

PAR ACA AED sde 


AAA T E E a aaa E 
IR SS ent PARAR AAA 
e f 3 
prp ns ad CET er DON 
DEET E kg A TE Eent, D 


















ENEE 



















ss dou M AAA L ‘ 
CET E a A sa STE bad S Me T Y TU 6 
Š AA E A AT E Pre et "» D 
pP ok € AM 










PrP DA 
PRE ET ET EE ii LAO 
Y PARAS 
A 1 TP AA] 


"1 SZT EU UE E 
DOT RED ITT s ñ 

A L E Weg DDT 
Deg T (D TUTE 











HL 
P A OA 










EE A end 
PET AAA ES ANA EI An 
l aro am T R L ara Pe Tel en a ell E AAA 
PRI A EE bpbro: t LTE NEG a O PETT TO PATA A 
o es RS EL and A DÉI mm CG E AE TE Taa E P D 
PE EE Feet EEN A A ral N AA NO E E TEA 
er ON perros TS E D oe E aay 
7 Moro TT ET Rd "TT III D k 
š d T ar amv? H 






AA mad pt aho d PTT 
TNE LIP 





m 
PETERE D TAE E 
































































r hrane ayq PSN 
kir brama T T el A —— Ñ NAAA OS 
PA a vp od. Epi n^ P eg ec Pd D p Per phere O MT api ' 
FE A SARA PII d LXI di Ir PEI J A TES PES AS TP PARRA 5 LIT eT e 
eben Er A mra gea ET AA Wy E AER AA] abet Beart be Š Z vR ei We de E De 
ARA EIA p pu AS puit ARA ELTE .. . o AO — — NE — ous $ L H 
— — 7 sk WE TE E GET E TEE A E Te id poroto D foots (ee AC E We c P de ee d . ail. , af (A " 16 
i i ARE T dr IRE Ree PE TT E TTE Harte de en e ? LIEU OT VJ D P 
Ho arm A E è PEE PNET EET Pree TP L A anager - " E S > RE E d 1 e .n. "AMT D " " 
AR Perd Au Pd eb m yy Here A T s: d " ear A — SHEET a NS 1 A 
AAN T S eE Kat, AI d E ER p mn PARA CHE * ar rii — MP sh d J r U Ze E D "E E 
ET AA E ima Ateh: o peanut,” AA AAA PRATS RO A A KEE EA rar ich ET ee : L — Ze r ° D " e 
ET ect EE Ed DIER x E "o" Y Pyra TA E d 2 N L IS. * d , 
Shair bri E E PE meld AEREA k af SAA FI haer ash IT PP — e ONO SE Bo, ES - n 
E lanla d par j A —* * POT TE P THP Ia? Lar A, = H k Ss coats et D ER tot - 
(Pda noL YY: prr "TM SOEN OH SÉ EA P ET > i —* d 
rad a a e — — O a Tn pup DN PRA PEE dde " "m ` E " 1 ET " P ', 1 š: d N Ë 
AA A E bd PETT Pt) a E O zt a io wi — L N : ' 

pue a A ër wee O BL M L E — Pë J 
IIA A A A So Pp dE DIESES TA A "EOM 30x. H m j N 
E L PAR AS ich wy IP LL PU TET NP WE EU e n. died 5 J^ d MESS SE, 0 D D) V 

ad ai A IT ISS AA TEER wem ot Lé (Ge Sgr " i V 
e Wb geg E, DEI WÉI. d La? (DH n A "P G 2 e É E ' D d 
d s . 
, ^ s ç. Tü L M 1,9! me n "rE Ag AE 
L4 

















TONS hie 
r$ (PR NL POE e 
EEN Nd a taba ie 
= ———— ouo PIS UE 
ei ENEE T EB E, 
ERAS 


PRAIRIE 
aar r a T" 
TUTTI LE 
P "PI 
m PM 
kada L M ad or de 
d erte 
dE orar 































PE ^ 
A E geg es 



































































































































































































































































































































































































































































































































































































































































PA ERA E iri a ri a E 
nper e ËTT E E Ee 
— — ma 
Pra d 
— la e 
E Pa 
— — — de SKS TR WE 
WW RA dete APA d 
INPEAP TE Dd AS A UE DES AC 
te: dr o TE Ted dia T kes, 
3 A de AAA 
e H H ER "255 * 
D S DEN " 
ick , 
[L4 
D 
IY ds d AR CAN 
AAA UR U L 
r P T A i l 
AA E N . 
— — s: nZ D 
* CARTA “°. 
AN s 2 
ATTE ide | 
DITA A 0 POTTS 
A E E E " 
et Se ee ped oA Ñ 
7 z z IN 
aye. 
mists Š à 
E 
LEE 35 344474 B^ 
- GË EE 
AA Ai H L 
LA ATT D 
— 
YP PIE 
CIA AI 
ATTE j 
PISO do n J 
AAA DA E v 
s 
q. TE 
> M D d 
N CAS 2 . 
AA aii L, 
L LAA T TT 
TRAJ nnan A Dn" y » AT S 
Ah Gw Pd tek Ae d TA LI H H A 
FIA 8. AAA HEITE KC ILAN 
Ge 1 dat 3 e Le WER EE a LN A De : 
he »* ebe Ad yst THELA FEE AS 
g^ GRN N * O WI (e CH. 
d'Ge EC Ce TS > Pia o TI TLS Q w 
US Xi I LÀ cy PEO TS efe A TD : 
CATAT aieia AS E m d M £ 
s“ ç 9 [N HLA y ° H . ,. SL FET] D 
KE a D Ah ER T ` - RM 
y SCH) Xs TA E A TY esie ° ata ? 
a HT T E KEES ON ` 1 IN N 
ALE ^ 4 e Ç HS E 
ELCH HLH KH ZA T " " H x 1 
OMNI NEL s 2 
u TOA P ^ d 
T Ex SN JK Pi +: * 
D ET] L] P 
`. WER: aa haft "in M NE 4 zg dere / - . 5 J 
H O x N T.E c e 3 ' 4 
WA Pr ta 
ROS AM i " M 
Tite oe és vajar tes, . M 
"UIDI m V a ET, s 
a bid A EH IU ICI d GZ 
KMA keete J; . AB (Be ' A ĩ 
— di e D TEE POCO LA "m : 
A EN ST TAR A O Gi d K j ; 
Lui LE D MME LA Qa t? Ad Mila. < . 4 ' k; 
Mad, (RSC TUTE ie ST LIT AEn + 1 e d 
LTE A kk IT Sp PEL, y D D e.” B 
NARRO < , GD j a Sie E gen v U D 
did rl PI US AP CT O H 4 $ .. >» 
ST O ES KU mz e * M ` N 
2: d Wa «cq * 9 ‘ "TI H L 4 D 
N ae a, aur? L ULL LO T m S UD d 
[UT RTE ESO TER — O a erh — APEA L n 34 ^o A Ñ T 
rider: ——— EE OA WE a O [ T M A 
D — — — das N TAS META A "T qe» k ba 1 à 
ea e Pk. — KTN o PT E E da PP 1 ' w. s 'Ó 
7 al + ga da — AO: «vda "* CNS D P 
"seeën erf ged Ee en Eae at ITU TER Ma me reer tot to ta) ke LIA — TPNT Y N OR: nun 
ATA IPLA MARIA GER OR SE i i 
EI NTT Td AA LA A] < k 14 8 DANI D 
CNA PTU DS LAIA NU O OTELI F ELEL š: A TI ue "` " 9 LI r L PON SN 
LAAL NAA NNN H AA ALS ' WO? Ya TT ET e ET L Pa f k d gd 
E fqw^u wie P mu PISO E S ? " KS n TL " rm Quoc P 
ITE qw, pront IES OCT da | ne. 1 D D .» D { b d 
T LH ND i. N KN LE Pr 1 Ft A CI A AN hac M t 
e BZ a M . OA LA DS Lo ah — TT 44 H HL AAA L on Wa HI D D LT! d y 
hpc pedo to dE CT AA Met ARA | et TY oa. eee 
Y —— 9 — RT TS ey AARI: O — l) ET he It t ^n deer. 
—— R YA Tenaga AAA IATA wa A E A T EC — ERC, c IORA ° 
Per Ey) ee ee) AA C NT AO PTT ——— Ñie ww < "°$ R D CL LOS? se es " 
de aid Acht A TIOS E MSC LR LER a: - IS 
A A 5 T L O V D dE ee 
Rég T KAUTH E . Ç a à 
uem = e TA — e EL 0. P 
h d A " d D 
A Soph tana rin PEE E z ar v ` V [E eh ` N nc 
Acre EXPO ote bel — — ta ARO TES H T D 1 Ms L c g y : * 
id Maru wt eere kad c PE A A lA 
DEL bo let dn M do bi EP ILLI E KR —— DT ia n IA s `: H 4 D 
CW E CN Fa OTTO D RR EL HRS CH AN va Pr TL ` WW LL D S 
ETA LULA ROLE Ok WA LEWECK OK WITT CRT AE ^ * Dedi GL 
FA A A XLI Leld de TL TA AA] “pe CA d . D RE u 
TE AA au OTI AE AAA > e n E u TIN O P 
AA be il L PN YW TSS TOS TIT y + - O » 
PTA O AHU LN TTT E Fa adaa, Bid Ce LC — " vlt L 
A er e CTT AL | eniro TA PER LONE d JR 
DOV K MO iat? LT dale eed E j "OM 3 y i 
F L IK? J $. b. e t 
T T TH f U tas YR wÑ eu Jr BN, dudes = 
AUT EL U Le x E DD TEL 8 í Wi A 
NLH dsd di ; LET IS ARENA 3! c " D M i 
daa AA ASTRAL NA. PTA de ET RAA H HOA MI f ° Aa VW 
AEB A ae IA eya Te toto é8 ete. rU tie eek ie te ke TETTE 1 e Yo 
AA d bi i na ols Li ted, Lh ke jy yt e vaseg t 3 gh ki I 10 TUE L n t A 
AT e A KEE LA id A de MI METE TOA MOES AE MS ba U IW 
jsx6s^ 4 9 9 AT mao do d VW gKOSN ARA f — E e 
KEE KI ECK A NM LE] d Fl ` 
E E TK CH ba Kal EXP ul H EDDA iA. N 14 Q N 
OA EM iE TERANA T ae PUL A N Ea 
— * E Te ag NER AER N TN t - | , n 
Li Late bdo adh WP Pv toe PETER MA ROZAS [E raTa al et D = 
UR aah — Déi? KEN Sak NA TUS A A] d NS NEU WTA WH E tt UE P 
"RT T liae AÑ NN re yi tr tee KAN GC LO Ka RS MC e ` ^ $ 4 
H ai k s LA Biada Lit SE D LA AA AT A AN A ae 3 U Ç > 
ao Ji dl e d PATA ed | pe ern th) * CORA ee MITA TIA , "a A N L E P 
AA ee A ET ML laa IR, dd LP L ` H š š > N 
e TITEL ULT AU dl CEN O MS KUY B M ELA L d š 
modern dA Tea TR AN TT E dd GK A F y Ze H DG SE A 
wa. Pd Fe o T “ë eri” n Tawa? ps 43, E d 15 
eate Ld ida * Ada LE! | , =" m ¢ š D 
I eid A erch AM Aa E ? " A N 
n LI P Era ert Ke, AE e WAAR e H 
ach — Ciad atii ES A A Ke WÄRE DS Ta 9 u D * iu 
Lid aiv ARA RT TTT : y š N V NEM NE Daf " a 
as o A KR sé A ATA LA A Dr mee TE AS TE oe . " 2 hos LL es L è 
TR EA hd ‘aw weer a Wich N AT Lu T 47 ut + " > 
A DA a A Sa Dr T AAA IRAN LE fer 8 
OGO -* ory ka Vat Pc 2 AA t r 
pu A O AA y 4 A v CH | RO t 
— — TLN Ki A, A EA RE CEO 
k LKA L.L. rT et DIA (LG iw REM OZ ki JL UU IMAN d SA FO TEL n e te — Li | +. f 
, AA Lb LE Le KD Ao upa QR: AU TETTE ve I Lu di BUMP. Md ` 
Lara od CR a da dd AA EA ai AAA ALA A Wr ek re " š y U A TT " 
SS MT A AS EE ALO A 2.06 nno NÉ UELLE EUER CEE E EITE U NEL. m ; L^ a 
— — rp p ty pv Ye lg °G aA Hr uh x A Pie A AA a RN H wt TUTA LKT SEGI AES TONA AAI m f 
— — — EM vg bane tH | v dad R LAH LAE S 1 (GUA [R + AX EH, NAIA Ly L R TR LE LW H Y i 
wrta dd KAA Y D Sa ad Ada LESE E E EA ECH hi 3 AA ADA NE ey AR A U LO m d Ta ` 
r TRE E KOL GC ed A Xv wv 279930 € dads nr ONG SEIL A, LLL LAE ven V ib, d BUT P P »*^ d IU H? 
peer TELE TAI degree CURA A AS vtr — Ne t. OCAK ASA Ka. A Et ci do M 29 PT 
(boc fei i aera RAQ EET TUS Lean » Di COIT TY SC q A YU — Tr 
^ LA za 1. EA LE (3 FT E] x — 75 1 N Pap v 144 c Ak E e E 
La a) dads De te See HECK A AA TOLE CE ETE ere v i à ; TA P H D q " 
e DESL OVA AA AN Ur A da ATI MN a DERE e A ^ d 
—— rS oe LLL d A SE MORE 3 E 0 dé 
Ae TG HUN TN EN STI AN e La TEEN CNA? KAAL H MAT ICA ^ 1 - WA - r 
ana AA ra'e AKALLA EA P ` — 
ep d e m. së (tf... V Wi 1 c ` " . [] U 
-I L 





